Exploring why the implicit regularization effects provided by neural networks can be effective for geophysical inversions

Recent research in test-time machine learning methods has shown that some machine learning models without any prior learning can improve the results of geophysical inversions. Some examples include the Deep Image Prior Inversions (DIP-Inv) and the Neural Fields Inversions (NFs-Inv), where the inverse problems are reparametrized by the weights of the machine learning models, and those weights are estimated during the inverse process. These methods utilize the implicit bias, which is inherent in the machine learning model structures, to impose a useful regularization effect on the geophysical inverse problems. However, the underlying mechanism of this implicit bias has not been fully explained. Recently, the generalization of modern supervised learning models in some inverse tasks in computer vision has been attributed to implicit bias as well. Considering that this implicit bias does not come from the training data set, we could utilize the analysis of that work to take one step further to explain the performance of the implicit bias and test-time learning methods in the geophysical inverse problems. In this work, we will show that the test-time machine learning methods can improve the geophysical inversion result by finding weights that can capture geometric structures in the physical property model (or so-called “geometry-adaptive harmonic bases”). We use neural fields, which use neural networks to map a coordinate to the corresponding physical property value at that coordinate, in a test-time learning manner. For a test-time learning method, the weights are learned during the inversion, as compared to traditional approaches which require a network be trained using a training data set. The test results for seismic tomography inversions and direct current resistivity inversions are shown first, followed by the eigen-decomposition analysis for both cases. The results show that the test-time learning approach can eliminate unwanted artifacts in the recovered subsurface physical property model caused by the sensitivity of the survey and physics; therefore, NFs-Inv improve the inversion results compared to the conventional inversion in some cases such as the recovery of the dip angle or the prediction of the boundaries of the main target. Our analysis will partly explain this phenomenon. Further works such as the applications in the field data and other theoretical analyses of the implicit bias are still in progress. By showing that the implicit bias brought by the Deep Neural Networks (DNNs) can benefit geophysical inversions, we also give insights into other machine learning methods in geophysics.