The I&D (Image & Diffraction) team of Dr. Arnaud Demortière (CR-CNRS) develops algorithmic tools based on artificial neural networks for electronic 2D / 3D image processing and X-rays for the study of properties dynamics of Li-ion battery materials.
Recent advances in electron microscopy (TEM) [1] and X-ray (STXM, XCT) [4] in terms of ultra-fast acquisition, high-resolution detection, 4D mapping technique and multidimensional analyzes open the way the way to a new horizon in the characterization of materials with the emergence of big data and AI processing tools. The generation of massive datasets through the operand and the multi-modalities requires the use of automated processing and statistical correlations, which are made accessible by the development of machine learning (ML) algorithms and deep learning (DL) [2]. ML and DL approaches have proven their great capacity in various fields of data processing, in particular in image processing, such as semantic segmentation [3], super resolution [4] and multimodal correlation [5].
Artificial neural network approach for multiphasic segmentation of Nano-CT images of battery electrodes [6].
The segmentation of the tomographic images of the battery electrode is a crucial processing step, which will have an additional impact on the results of material characterization and electrochemical simulation. However, manual labeling of X-ray computed tomography (XCT) images takes time, and these XCT images are generally difficult to segment with histographic methods. Here we propose a deep learning approach with an asymmetric depth encoder-decoder convolutional neural network (CNN) for real-world battery material datasets. This network achieves high accuracy while requiring small amounts of tagged data and predicts a volume of billions of voxels within minutes. To apply supervised machine learning to segment real world data, ground truth is needed and therefore the segmentation results are usually qualitatively justified by visual judgment. We unravel this fuzzy definition of quality segmentation by identifying the uncertainty due to diluted human bias in the training data. Training continues with synthetic data to show the quantitative impact of such uncertainty on the determination of material properties. Nano-XCT datasets of various battery materials have been successfully segmented by training this neural network from scratch. We have also shown that the application of transfer learning, which involves reusing a well-trained network, can improve the accuracy of a similar dataset.
Self-supervised image quality assessment for X-ray tomographic images of Li-ion battery materials [7].
Image perception plays a fundamental role in tomography-based approaches for characterizing microstructure and has a profound impact on all subsequent stages of image processing such as segmentation and 3D analysis. Improving image perception, however, frequently involves observer dependence, which reflects user-to-user dispersion and uncertainties in calculated parameters. This work presents an objective quantitative method, which uses convolutional neural networks (CNN), for the evaluation of the quality of the X-ray tomographic image. With only dozens of annotations, our method makes it possible to directly evaluate and precisely the quality of the tomographic image. Different metrics were used to assess the correlation between our predicted scores and subjective human annotations. The results of the evaluation demonstrate that our method can be a direct tool to guide the improvement process with the aim of producing reliable segmentation results. The processing of the tomographic image can thus evolve towards a robust procedure independent of the observer and advance towards the development of an efficient self-supervised approach.
Compressed sensing for ultra-fast image processing in liquid media.
The monitoring of the dynamic phenomena of charge / discharge of battery materials is made possible by the acquisition of ultra-fast TEM images (and diffraction) in liquid. The constraints linked to rapid imaging and to the minimization of the effects of irradiation lead to the development of new approaches such as “compressed detection” (CS). The CS makes it possible to combine detection and compression in a single operation, improving the acquisition speed and reducing the electron dose. This involves recovering the signal from sparse measurements using a CS approach coupled to an ANN network (GAN). We propose in this part to adapt the "compressed detection" approach allowing to combine detection and compression in a single operation in order to improve the acquisition speed and reduce the electron dose in TEM experiments in liquid medium. Compressed detection is a signal processing technique for efficiently acquiring and reconstructing a signal, finding solutions to under-determined linear systems. This is based on the principle that, through optimization, the parsimony of a signal can be exploited to recover it from much fewer samples than required by the Nyquist-Shannon sampling theorem [4]. As shown in Figure 2a, recovery of the HR-TEM image signal (100%) from sparse measurements (20%) is made possible by this approach. We propose here to combine “compressed detection” with GAN (Generative Adversarial Network) neural networks to allow greater flexibility of use in order to improve reconstruction performance. This approach will be used to perform liquid TEM cell measurements with minimal acquisition time and low beam, which will minimize the effects of electron dose. This approach can be used for reconstructions of images and diffraction patterns.
Deep learning approach for automatic identification of diffraction pattern from 4D-STEM data.
The latest generations of characterization tools based on electronic diffraction (4D-STEM) and rapid imaging allow the collection of a large volume of data requiring the development of automatic image processing following the deep learning approach. For 5 years, at LRCS, we have been developing the characterization of Li / Na-ion battery materials by orientation and crystalline phase mapping based on electron diffraction. In this project, we will work on the development of an approach for the automatic identification of diffraction images based on the use of deep learning networks. Based on our experience with artificial CNN networks (programming in Python + TensorFlow) for image segmentation in X-ray tomography and automatic image quality assessment, we will set up a "workflow" allowing the processing. automatic operation of a large amount of data from 4D-STEM acquisition. This will involve working on a database of diffraction images for the stages of learning networks with the help of the company NanoMegas (project partner) to access a large base of 4D STEM images. It will be a question of considering all the symmetry groups and the different possible orientations of the analyzed crystals, starting with the crystals of NVPF, LMNO and LFP as reference materials. We will also be able to rely on open access databases [3]. The optimization of the identification of the patterns in the diffraction images will be done according to the coupling of 2 approaches which will lead to the establishment of the different classes: one, with the construction of geometric pattern according to criteria of distances and periodicity , and the other, taking into account the intensity of the spots. The presence of defects such as vacancies, interstitials, anti-sites or dislocations could also be taken into account to improve the model. The architecture and dimension of the convolutional neural network will be optimized and associated with the DenseNet multistream to obtain high classification precision.
Multi-scale and multi-modal data processing.
In this project, we will build and develop tools to help the user manage a large dataset and process the data automatically (supervised approach) and use machine learning and deep learning approaches. (a) Data cleaning, recording and segmentation. Raw image, spectrum and diffractogram data should be processed to reduce experimental defects such as noise, blur artifacts, rings, variation in intensity as they have strong effects in the next step of the process . A generalization of our recent work on unsupervised image quality assessment could be useful for this purpose. The recording step, that is to say the realignment of the image in the time series, must be carried out and verified. Then the multiphase segmentation will be performed using our in-house SegmentPy software based on the CNN approach. (b) Multimodal correlation process. To predict and interpret a complex hierarchical microstructure, a joint knowledge of their multiphase properties is necessary to have a good interpretation of the properties of the battery microstructure. A CNN-based similarity learning network coupled to an unsampled shearlet transform (NSST) for the decomposition of images, spectra and diffractograms into low and high frequency components will be the basis of our strategy. For imaging, the fusion strategy will also be used to generate a mixed multimodal image increasing the interpretability of the dataset. (c) Multiscale correlation process. This step will focus on finding correlations and patterns in features present in images at different scales. To this end, ANN networks will be used to link scales and connect models from one scale to another using a super-resolution network (SRCNN). Tools such as the residual network and the hop connection will be considered to improve the efficiency and accuracy of the SRCNN for this purpose. (d) Dynamic correlation process. From the prepared time series dataset, the dynamic process description strategy will be designed with an RNN network consisting of LSTM (or GRU) units. The optimization steps will be performed on the structure and hyperparameters of the RNN network. Data learning will follow a separate and mixed strategy for each subscale. Multimodal fusion images could be tested as a second step. Particular work will be carried out to identify the relevant loss, ground truth and metric functions suitable for each type of data in order to best reflect the dynamics observed. Alternative strategies in terms of network, such as the convolutional RNN and the Prod-RNN will also be tested to take into account the spatio-temporal aspects joining the spatial correlations and the temporal dynamics at different levels and combining the advantages of the CNN and RNN architectures.
[1] Liu, X., & Gu, L. (2018). Advanced transmission electron microscopy for electrode and solid‐ electrolyte materials in lithium‐ion batteries. Small Methods, 2(8), 1800006.
[2] Nelson, J., Misra, S., Yang, Y., Jackson, A., Liu, Y., Wang, H., Toney, M. F. (2012). In operando X-ray diffraction and transmission X-ray microscopy of lithium sulfur batteries. Journal of the American Chemical Society, 134(14), 6337-6343.
[3] Hao, S., Zhou, Y., & Guo, Y. (2020). A brief survey on semantic segmentation with deep learning. Neurocomputing, 406, 302-321.
[4] Wang, Z., Chen, J., & Hoi, S. C. (2020). Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence.
[5] Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., & Ng, A. Y. (2011, January). Multimodal deep learning. In ICML.
[6] Artificial Neural Network Approach for Multiphase Segmentation of Battery Electrode Nano-CT Images, Zeliang Su, Etienne Decencière, Tuan-Tu Nguyen, Kaoutar El-Amiry, Vincent De Andrade, Alejandro A. Franco, Arnaud Demortière, NaturePJ computational materials , 2021 (in press)
[7] Kai Zhang, Tuan-Tu Nguyen, Arnaud Demortière, Self-supervised image quality assessment for X-ray tomographic images of Li-ion battery materials, NaturePJ computational materials, 2021 ( in reviewing)