Machine learning in materials engineering

Advantages of data-driven methods

Data-driven methods are increasingly finding their way into materials science. Applications range from material design, identification of hidden relations (data mining), and prediction of material’s properties to characterization of microstructures, defects, and damage. By using such methods, predictions can often be greatly improved compared to purely knowledge-based approaches due to their ability to generate highly accurate representations and high fidelity. This is especially true for many problems for which domain knowledge and the models based on it are incomplete.

to top

Limitations of data-driven methods in materials engineering

Conventional machine learning approaches require a large amount of annotated training data for so-called supervised learning. Appropriate methods are characterized by low data efficiency regarding learning compared to humans. Since the data annotation process can often only be performed by experts and is correspondingly expensive, the high demand for such methods in materials science cannot be readily satisfied. In addition, many models have poor transferability across data sets. This lack of generalizability can also be ascribed to sparsely populated training data spaces that contrast large parameter spaces in materials development. The combination of the previously mentioned characteristics makes prediction for new materials difficult.

to top

Finding solutions through new methods in machine learning

Within the Meso- and Micromechanics group at the Fraunhofer Institute for Mechanics of Materials IWM, one focal point is dealing with qualifying novel data-driven approaches as well as optimizing these approaches in order to address both of the previously mentioned challenges.

In the course of the last years methods have been adapted which make use of annotated data as well as unannotated data in order to significantly improve predictions compared to supervised learning methods - despite the shortage of annotated data. These methods are subsumed under the term semi-supervised learning methods. Unannotated data can often be collected without much additional effort thanks to the high automation level of industrial plants and analytical methods.

We address the aspect of transferability between domains with models that make use of annotated data from a source domain and unannotated data from a related target domain to achieve good prediction performance in the target domain. This task is called unsupervised domain adaptation. Different process control, modalities, materials, and data sources (e.g., synthetic/simulation data gleaned from experimental data) serve both as source and target domains.

The amount of data required depends on the complexity of the neural network architecture and the concrete problem. By means of intelligent strategies for data augmentation adapted to the problem, as little as ten annotated image instances can be sufficient for training relatively compact networks.

to top

What we offer

By combining competencies from materials science and computer science, we train customized models for you. In doing so, we take relevant data features into account. Regarding this, we can support you in the entire process chain from the appropriate data collection to the optimal data preparation and model selection as well as training, evaluation and deployment. 

  • Consultation and/or support during data acquisition
  • Appropriate preparation (scaling, normalization, tiling, registration...) of data considering relevant features and model characteristics as well as their optimal data augmentation
  • Selection of suitable data for pre-training from our internal material data pool to improve models despite data shortage
  • Model selection and implementation
  • Classification, segmentation (binary, semantic, instance, panoptic), and regression tasks on pixel data, voxel data, tabular and graph-structured data
  • Supervised training of different (pre-trained) models using high-performance hardware for faster results
  • Semi-supervised training with additional help of unannotated data for better models in case of low data quantity
  • Unsupervised or semi-supervised domain adaptation between related materials, processes or modalities can help to utilize your existing data for new target states without annotating new data.
  • Model evaluation and robustness testing

to top

Example applications

© Fraunhofer IWM
Fig. 1: Segmentation results of models trained with annotated light microscopy and scanning electron microscopy data under different training conditions: (a-b) randomly initialized conventional U-Net, (c-d) randomly initialized VGG16 U-Net, (e-f) ImageNet pre-trained VGG16 U-Net. Lath-shaped regions are well detected after supervised training in the target domain.

Microstructure quantification of complex phase steels

 

In two collaborations, we have developed image segmentation models that can identify lath-shaped bainite phases in a complex-phase steel. For this purpose, light and scanning electron microscopic images of etched (for contrasting) surfaces were used. If this task is handled by common knowledge-based models, the prediction quality is comparatively poor. To address this complicated problem, we use convolutional neural networks (FCNN). In the course of the previously mentioned research work, we systematically developed guidelines for the application of different networks depending on the data. Strategies for data augmentation were derived to optimally take data scatter into account. Furthermore, the issue of interpretability of FCNN models was addressed with various visualization methods to increase confidence in the models and to examine them for spurious correlations. The supervised trained models have achieved Intersection over Union (IoU) values of about 80% (this corresponds to accuracies of about 90%) on the test data for different modalities, see Figure 1.

The IoU metric is defined between 0 and 100% and is determined using the number of true positive, false positive and false negative pixels:

IoU = TP/(TP+FP+FN)

© Fraunhofer IWM
Fig. 2: Dataset description for domain transferability studies with micrograph, micrograph overlaid with the annotation mask, and a detail image. (a-c) Source domain (S) consisting of scanning electron microscopy images, (d-f) target domain 1 (T1) with an alternative etch for microstructure contrast, (g-i) target domain 2 (T2) with alternative and overetched microstructure contrast, and (j-l) target domain 3 (T3) with light microscopy data.

These accuracies correspond to a segmentation quality achieved by metallographers, but offer the advantage that even complicated microstructures can be segmented automatically and reproducibly (more information). However, when supervised trained models were transferred to another etching, these models failed.

In order to address the poor domain transferability of many FCNN architectures, a framework for unsupervised domain adaptation (UDA) has been explored. This problem deals with the case where there is no annotated data, but only input data for target domains, and a good segmentation should be achieved in the target domain. Two different etchings and light microscopic images were used as target domains for this study (see Figure 2).

By using UDA, the mean Intersection over Union (mIoU) was improved over the baseline models (supervised trained on source domain and applied to target domain) from 82.2%, 61.0%, 49.7% to 84.7% (+2.5%), 67.3% (+6.3%), 73.3% (+23.6%) for the three domain transitions, respectively. The comparison between baseline models and UDA is shown in Figures 3-5.

This highlights the potential of these models to develop data-driven methods that can handle different domain distances well, and could significantly reduce the need for annotation (more information).

© Fraunhofer IWM
Fig. 3: Predictions for the test images of target domain 1. The left column (a), (c) shows the predictions of a DeepLabv2 model which was trained on the source dataset. The right column (b), (d) shows the predictions of a DeepLabv2 model which was trained with the UDA framework using the source and target 1 datasets (the latter without annotations).
© Fraunhofer IWM
Fig. 4: Predictions for the test images of target domain 2. The left column (a), (c) shows the predictions of a DeepLabv2 model which was trained on the source dataset. The right column (b), (d) shows the predictions of a DeepLabv2 model which was trained with the UDA framework using the source and target 2 datasets (the latter without annotations).
© Fraunhofer IWM
Fig. 5: Predictions for the test images of target domain 3. The left column (a), (c) shows the predictions of a DeepLabv2 model which was trained on the source dataset. The right column (b), (d) shows the predictions of a DeepLabv2 model which was trained with the UDA framework using the source and target 3 datasets (the latter without annotations).

Image segmentation of material damage (cracks and extrusions)

© Fraunhofer IWM
Fig. 6: Cracks and extrusions from scanning electron microscopy (SEM) of different materials: a) ferritic steel EN 1.4003, b) martensitic steel EN 1.7228 and c) oxygen-free copper.
© Fraunhofer IWM
Fig. 7: Extrusion and crack in ferritic steel EN 1.4003 overlaid with segmentation masks showing background (red), cracks (green) and extruded regions (blue). a) Manually indexed regions, b) Neural network prediction.

Another issue within data-driven image processing is the segmentation of fatigue damage [1]. The condition of such damage is highly dependent on the material (Figure 6) and the loading scenario. In addition, image data of damage, due to optical imaging settings, exhibit scatter. Machine learning methods allow automated detection of such damaged locations in diverse materials. Thus, pixel-by-pixel crack and extrusion detection (semantic segmentation) can be performed using neural networks that have already been trained with a versatile data set. Since the training dataset contains a wide variety of crack and extrusion morphologies (see Figure 6) such as protrusions 1) and localized tongue-shaped extrusions 2) in several materials, it is assumed that this method can be transferred to other metallic materials.  We combine this automated and generalized method with automated scanning electron microscopy (SEM) to provide efficient damage characterization on sample and component surfaces. For example, in damage detection (Figure 7) in EN 1.4003 ferritic steel, a mean IoU for cracks and extrusions of 0.84 and 0.71 was obtained, respectively. Such damage segmentation has been used, for example, to statistically validate crystal plasticity simulations for short crack growth [2].

 

[1] Thomas, A.; Durmaz, A. R.; Straub, T.; Eberl, C., Automated quantitative analyses of fatigue induced surface damage by deep learning, Materials 13/15 (2020) Art. 3298, 24 Seiten Link

[2] Natkowski, E.; Durmaz, A.R.; Sonnweber-Ribic, P.; Munstermann, S., Fatigue lifetime prediction with a validated micromechanical short crack model for the ferritic steel EN 1.4003, International Journal of Fatigue 152 (2021) Art. 106418; 15 Seiten

 

to top

Publications

  • Natkowski, E.; Durmaz, A.R.; Sonnweber-Ribic, P.; Munstermann, S., Fatigue lifetime prediction with a validated micromechanical short crack model for the ferritic steel EN 1.4003, International Journal of Fatigue 152 (2021) Art. 106418; 15 Seiten
  • Thomas, A.; Durmaz, A. R.; Straub, T.; Eberl, C., Automated quantitative analyses of fatigue induced surface damage by deep learning, Materials 13/15 (2020) Art. 3298, 24 Seiten Link

to top