Find a suitable baseline problem

[pzimbrod]

We need a solid foundation in form of a script that covers the inference of (material) properties from an inverse problem.

For starters, looking into the DeepXDE example problems is probably a good idea

[MichaelFlec]

I think the inverse problem with the diffusion reaction equation is a good starting point: diffusion_1d_inverse.py

However, yesterday I tryed out executing this python-script in my working copy of DeepXDE. Unfortunately, it failed with the message:

  File "diffusion_1d_inverse.py", line 10, in <module>
    C = dde.Variable(2.0)
AttributeError: module 'deepxde' has no attribute 'Variable'

This missing attribute has been added very recently to DeepXDE, so it seams as if it is only not contained in my active version. Nevertheless, by updateing with conda I could not fixed the problem.

[pzimbrod]

I ran this script up to line 10 that causes problems on your system and did not experience issues. I previously updated my environment using conda update deepxde today since I was using an older Version of DeepXDE as well.

Alternatively, you could always set up an entirely new conda environment using conda create --name my_env, activate it and do a fresh install of DeepXDE.

Nonetheless, I'll investigate the case you proposed soon. I pushed the script and the data into the branch that goes along with this issue. Running the script from the main dir, training of the model started without errors on my machine on CPU.

[pzimbrod]

I have tried our sample case using the PyTorch Backend on the GPU (NVIDIA Quadro RTX A5000, 8k+ CUDA Cores) and got these results:

GPU Training

```bash ╭─    ~/ML-for-PhaseField  on   pzimbrod/issue1  ╰─ DDEBACKEND=pytorch python models/diffusion_1d_inverse.py ─╯ Using backend: pytorch Warning: 50000 points required, but 50055 points sampled. Compiling model... 'compile' took 0.000084 s Training model... Step Train loss Test loss Test metric 0 [1.14e-02, 7.91e-03, 2.52e-01, 3.61e-01, 2.52e-02, 2.52e-02, 1.57e-01, 5.75e-02] [1.08e-02, 7.44e-03, 2.52e-01, 3.61e-01, 2.52e-02, 2.52e-02, 1.57e-01, 5.75e-02] [] 1000 [1.83e-04, 2.38e-04, 3.32e-04, 2.66e-04, 5.15e-04, 4.26e-04, 3.18e-04, 1.87e-04] [1.34e-04, 1.86e-04, 3.32e-04, 2.66e-04, 5.15e-04, 4.26e-04, 3.18e-04, 1.87e-04] [] 2000 [5.56e-05, 4.96e-05, 6.11e-05, 5.85e-05, 7.87e-05, 7.81e-05, 8.67e-05, 1.11e-04] [4.34e-05, 3.93e-05, 6.11e-05, 5.85e-05, 7.87e-05, 7.81e-05, 8.67e-05, 1.11e-04] [] 3000 [4.65e-05, 3.94e-05, 2.93e-05, 3.26e-05, 3.38e-05, 3.22e-05, 6.20e-05, 9.96e-05] [4.10e-05, 3.45e-05, 2.93e-05, 3.26e-05, 3.38e-05, 3.22e-05, 6.20e-05, 9.96e-05] [] 4000 [4.19e-05, 3.59e-05, 1.79e-05, 2.56e-05, 2.26e-05, 2.11e-05, 4.50e-05, 8.23e-05] [3.82e-05, 3.16e-05, 1.79e-05, 2.56e-05, 2.26e-05, 2.11e-05, 4.50e-05, 8.23e-05] [] 5000 [3.87e-05, 3.35e-05, 1.43e-05, 1.34e-05, 1.75e-05, 1.66e-05, 3.30e-05, 8.19e-05] [3.59e-05, 2.90e-05, 1.43e-05, 1.34e-05, 1.75e-05, 1.66e-05, 3.30e-05, 8.19e-05] [] 6000 [3.70e-05, 3.12e-05, 9.97e-06, 1.34e-05, 1.36e-05, 1.27e-05, 2.96e-05, 7.09e-05] [3.46e-05, 2.68e-05, 9.97e-06, 1.34e-05, 1.36e-05, 1.27e-05, 2.96e-05, 7.09e-05] [] 7000 [3.49e-05, 2.87e-05, 1.35e-05, 1.23e-05, 1.29e-05, 1.13e-05, 3.38e-05, 6.73e-05] [3.29e-05, 2.46e-05, 1.35e-05, 1.23e-05, 1.29e-05, 1.13e-05, 3.38e-05, 6.73e-05] [] 8000 [3.27e-05, 2.62e-05, 7.21e-06, 1.35e-05, 8.86e-06, 8.49e-06, 2.47e-05, 6.04e-05] [3.10e-05, 2.24e-05, 7.21e-06, 1.35e-05, 8.86e-06, 8.49e-06, 2.47e-05, 6.04e-05] [] 9000 [3.03e-05, 2.43e-05, 5.29e-06, 9.14e-06, 7.80e-06, 7.45e-06, 2.01e-05, 5.97e-05] [2.89e-05, 2.05e-05, 5.29e-06, 9.14e-06, 7.80e-06, 7.45e-06, 2.01e-05, 5.97e-05] [] 10000 [2.81e-05, 2.28e-05, 4.73e-06, 9.93e-06, 6.76e-06, 6.63e-06, 1.78e-05, 5.47e-05] [2.70e-05, 1.92e-05, 4.73e-06, 9.93e-06, 6.76e-06, 6.63e-06, 1.78e-05, 5.47e-05] [] 11000 [2.60e-05, 2.19e-05, 3.96e-06, 7.13e-06, 6.15e-06, 6.11e-06, 1.45e-05, 5.25e-05] [2.51e-05, 1.83e-05, 3.96e-06, 7.13e-06, 6.15e-06, 6.11e-06, 1.45e-05, 5.25e-05] [] 12000 [2.40e-05, 2.13e-05, 3.52e-06, 8.34e-06, 5.61e-06, 5.56e-06, 1.20e-05, 4.67e-05] [2.33e-05, 1.76e-05, 3.52e-06, 8.34e-06, 5.61e-06, 5.56e-06, 1.20e-05, 4.67e-05] [] 13000 [2.19e-05, 2.08e-05, 3.20e-06, 6.13e-06, 5.03e-06, 5.21e-06, 9.42e-06, 4.32e-05] [2.14e-05, 1.70e-05, 3.20e-06, 6.13e-06, 5.03e-06, 5.21e-06, 9.42e-06, 4.32e-05] [] 14000 [2.05e-05, 2.03e-05, 9.28e-06, 2.85e-05, 4.50e-06, 6.49e-06, 9.14e-06, 3.43e-05] [2.00e-05, 1.70e-05, 9.28e-06, 2.85e-05, 4.50e-06, 6.49e-06, 9.14e-06, 3.43e-05] [] 15000 [1.82e-05, 2.05e-05, 2.64e-06, 4.17e-06, 4.06e-06, 4.45e-06, 5.46e-06, 3.42e-05] [1.78e-05, 1.64e-05, 2.64e-06, 4.17e-06, 4.06e-06, 4.45e-06, 5.46e-06, 3.42e-05] [] 16000 [1.64e-05, 2.07e-05, 2.99e-06, 2.47e-06, 4.15e-06, 4.10e-06, 4.49e-06, 3.14e-05] [1.60e-05, 1.62e-05, 2.99e-06, 2.47e-06, 4.15e-06, 4.10e-06, 4.49e-06, 3.14e-05] [] 17000 [1.50e-05, 2.07e-05, 6.18e-06, 5.92e-06, 4.09e-06, 3.82e-06, 5.67e-06, 2.40e-05] [1.45e-05, 1.64e-05, 6.18e-06, 5.92e-06, 4.09e-06, 3.82e-06, 5.67e-06, 2.40e-05] [] 18000 [1.35e-05, 2.13e-05, 1.07e-05, 7.58e-06, 4.14e-06, 4.90e-06, 6.34e-06, 3.28e-05] [1.31e-05, 1.59e-05, 1.07e-05, 7.58e-06, 4.14e-06, 4.90e-06, 6.34e-06, 3.28e-05] [] 19000 [1.21e-05, 2.08e-05, 1.82e-06, 1.89e-06, 2.86e-06, 3.07e-06, 2.44e-06, 1.92e-05] [1.17e-05, 1.59e-05, 1.82e-06, 1.89e-06, 2.86e-06, 3.07e-06, 2.44e-06, 1.92e-05] [] 20000 [1.08e-05, 2.03e-05, 7.35e-06, 7.27e-06, 3.36e-06, 2.80e-06, 4.63e-06, 1.35e-05] [1.04e-05, 1.57e-05, 7.35e-06, 7.27e-06, 3.36e-06, 2.80e-06, 4.63e-06, 1.35e-05] [] 21000 [9.73e-06, 2.03e-05, 5.57e-06, 2.24e-06, 3.69e-06, 2.75e-06, 4.79e-06, 1.63e-05] [9.32e-06, 1.48e-05, 5.57e-06, 2.24e-06, 3.69e-06, 2.75e-06, 4.79e-06, 1.63e-05] [] 22000 [8.64e-06, 1.94e-05, 1.48e-06, 1.62e-06, 2.12e-06, 2.34e-06, 1.83e-06, 1.03e-05] [8.22e-06, 1.43e-05, 1.48e-06, 1.62e-06, 2.12e-06, 2.34e-06, 1.83e-06, 1.03e-05] [] 24000 [6.71e-06, 1.72e-05, 1.35e-06, 1.36e-06, 1.91e-06, 1.98e-06, 1.60e-06, 6.20e-06] [6.30e-06, 1.20e-05, 1.35e-06, 1.36e-06, 1.91e-06, 1.98e-06, 1.60e-06, 6.20e-06] [] 25000 [5.83e-06, 1.56e-05, 1.32e-06, 1.18e-06, 1.81e-06, 1.83e-06, 1.43e-06, 4.64e-06] [5.44e-06, 1.05e-05, 1.32e-06, 1.18e-06, 1.81e-06, 1.83e-06, 1.43e-06, 4.64e-06] [] 26000 [5.25e-06, 1.35e-05, 1.72e-05, 1.57e-05, 4.65e-06, 2.25e-06, 7.43e-06, 3.36e-06] [4.82e-06, 9.36e-06, 1.72e-05, 1.57e-05, 4.65e-06, 2.25e-06, 7.43e-06, 3.36e-06] [] 27000 [4.20e-06, 1.15e-05, 1.39e-06, 1.02e-06, 1.70e-06, 1.57e-06, 1.18e-06, 2.50e-06] [3.83e-06, 6.83e-06, 1.39e-06, 1.02e-06, 1.70e-06, 1.57e-06, 1.18e-06, 2.50e-06] [] 28000 [3.57e-06, 9.78e-06, 1.29e-06, 7.84e-07, 1.71e-06, 1.53e-06, 1.11e-06, 2.12e-06] [3.22e-06, 5.50e-06, 1.29e-06, 7.84e-07, 1.71e-06, 1.53e-06, 1.11e-06, 2.12e-06] [] 29000 [3.72e-06, 1.00e-05, 8.12e-05, 1.05e-04, 1.42e-05, 1.35e-05, 3.85e-05, 4.10e-05] [3.33e-06, 4.00e-06, 8.12e-05, 1.05e-04, 1.42e-05, 1.35e-05, 3.85e-05, 4.10e-05] [] 30000 [2.80e-06, 7.14e-06, 1.09e-06, 7.44e-07, 1.65e-06, 1.33e-06, 9.42e-07, 1.49e-06] [2.48e-06, 3.77e-06, 1.09e-06, 7.44e-07, 1.65e-06, 1.33e-06, 9.42e-07, 1.49e-06] [] 31000 [2.56e-06, 6.17e-06, 1.06e-06, 7.02e-07, 1.63e-06, 1.26e-06, 8.59e-07, 1.31e-06] [2.25e-06, 3.20e-06, 1.06e-06, 7.02e-07, 1.63e-06, 1.26e-06, 8.59e-07, 1.31e-06] [] 32000 [2.38e-06, 5.25e-06, 1.91e-06, 1.64e-06, 1.74e-06, 1.24e-06, 1.02e-06, 1.26e-06] [2.07e-06, 2.80e-06, 1.91e-06, 1.64e-06, 1.74e-06, 1.24e-06, 1.02e-06, 1.26e-06] [] 33000 [2.20e-06, 4.73e-06, 9.40e-07, 6.75e-07, 1.59e-06, 1.14e-06, 7.51e-07, 9.59e-07] [1.92e-06, 2.37e-06, 9.40e-07, 6.75e-07, 1.59e-06, 1.14e-06, 7.51e-07, 9.59e-07] [] 34000 [2.05e-06, 4.19e-06, 9.09e-07, 6.28e-07, 1.55e-06, 1.08e-06, 6.98e-07, 8.41e-07] [1.78e-06, 2.08e-06, 9.09e-07, 6.28e-07, 1.55e-06, 1.08e-06, 6.98e-07, 8.41e-07] [] 35000 [1.92e-06, 3.74e-06, 9.76e-07, 8.02e-07, 1.53e-06, 1.06e-06, 7.42e-07, 8.35e-07] [1.66e-06, 1.77e-06, 9.76e-07, 8.02e-07, 1.53e-06, 1.06e-06, 7.42e-07, 8.35e-07] [] 36000 [1.82e-06, 3.27e-06, 1.09e-06, 7.04e-07, 1.51e-06, 9.77e-07, 6.66e-07, 6.66e-07] [1.56e-06, 1.65e-06, 1.09e-06, 7.04e-07, 1.51e-06, 9.77e-07, 6.66e-07, 6.66e-07] [] 37000 [1.77e-06, 2.91e-06, 6.24e-06, 2.32e-06, 3.76e-06, 1.06e-06, 3.54e-06, 8.38e-07] [1.51e-06, 1.60e-06, 6.24e-06, 2.32e-06, 3.76e-06, 1.06e-06, 3.54e-06, 8.38e-07] [] 38000 [1.60e-06, 2.72e-06, 8.57e-07, 5.61e-07, 1.40e-06, 8.84e-07, 5.34e-07, 4.50e-07] [1.38e-06, 1.30e-06, 8.57e-07, 5.61e-07, 1.40e-06, 8.84e-07, 5.34e-07, 4.50e-07] [] 39000 [1.50e-06, 2.48e-06, 7.16e-07, 5.16e-07, 1.32e-06, 8.30e-07, 4.93e-07, 3.90e-07] [1.29e-06, 1.17e-06, 7.16e-07, 5.16e-07, 1.32e-06, 8.30e-07, 4.93e-07, 3.90e-07] [] 40000 [1.41e-06, 2.27e-06, 6.86e-07, 5.04e-07, 1.27e-06, 7.79e-07, 4.99e-07, 3.75e-07] [1.21e-06, 1.07e-06, 6.86e-07, 5.04e-07, 1.27e-06, 7.79e-07, 4.99e-07, 3.75e-07] [] 41000 [1.33e-06, 2.10e-06, 7.44e-07, 5.34e-07, 1.28e-06, 8.19e-07, 4.54e-07, 4.18e-07] [1.14e-06, 9.87e-07, 7.44e-07, 5.34e-07, 1.28e-06, 8.19e-07, 4.54e-07, 4.18e-07] [] 42000 [1.24e-06, 1.99e-06, 5.93e-07, 5.20e-07, 1.18e-06, 7.33e-07, 4.22e-07, 2.82e-07] [1.07e-06, 9.41e-07, 5.93e-07, 5.20e-07, 1.18e-06, 7.33e-07, 4.22e-07, 2.82e-07] [] 43000 [1.31e-06, 2.32e-06, 8.03e-06, 4.78e-05, 4.59e-06, 1.73e-05, 3.50e-06, 2.43e-05] [1.11e-06, 8.88e-07, 8.03e-06, 4.78e-05, 4.59e-06, 1.73e-05, 3.50e-06, 2.43e-05] [] 44000 [1.10e-06, 1.77e-06, 5.64e-07, 4.07e-07, 1.06e-06, 6.29e-07, 3.22e-07, 2.28e-07] [9.51e-07, 8.41e-07, 5.64e-07, 4.07e-07, 1.06e-06, 6.29e-07, 3.22e-07, 2.28e-07] [] 45000 [1.04e-06, 1.65e-06, 7.05e-07, 6.13e-07, 1.18e-06, 7.67e-07, 3.43e-07, 5.16e-07] [8.99e-07, 8.11e-07, 7.05e-07, 6.13e-07, 1.18e-06, 7.67e-07, 3.43e-07, 5.16e-07] [] 46000 [1.00e-06, 1.60e-06, 9.66e-06, 2.24e-06, 6.33e-06, 1.31e-06, 7.25e-06, 1.06e-06] [8.66e-07, 7.51e-07, 9.66e-06, 2.24e-06, 6.33e-06, 1.31e-06, 7.25e-06, 1.06e-06] [] 47000 [9.36e-07, 1.57e-06, 2.52e-06, 3.98e-06, 4.38e-06, 3.69e-06, 3.72e-06, 3.32e-06] [8.07e-07, 8.16e-07, 2.52e-06, 3.98e-06, 4.38e-06, 3.69e-06, 3.72e-06, 3.32e-06] [] 48000 [1.28e-06, 1.98e-06, 4.19e-05, 2.75e-05, 1.50e-05, 1.51e-06, 2.49e-05, 6.77e-06] [1.07e-06, 9.07e-07, 4.19e-05, 2.75e-05, 1.50e-05, 1.51e-06, 2.49e-05, 6.77e-06] [] 49000 [8.51e-07, 1.42e-06, 4.86e-07, 4.14e-07, 8.56e-07, 4.76e-07, 2.71e-07, 2.27e-07] [7.33e-07, 7.15e-07, 4.86e-07, 4.14e-07, 8.56e-07, 4.76e-07, 2.71e-07, 2.27e-07] [] 50000 [8.06e-07, 1.38e-06, 3.99e-07, 3.34e-07, 8.01e-07, 4.51e-07, 2.00e-07, 1.71e-07] [6.95e-07, 7.07e-07, 3.99e-07, 3.34e-07, 8.01e-07, 4.51e-07, 2.00e-07, 1.71e-07] [] 51000 [7.72e-07, 1.33e-06, 3.86e-07, 3.13e-07, 7.67e-07, 4.26e-07, 1.80e-07, 1.68e-07] [6.65e-07, 6.88e-07, 3.86e-07, 3.13e-07, 7.67e-07, 4.26e-07, 1.80e-07, 1.68e-07] [] 52000 [7.39e-07, 1.29e-06, 3.76e-07, 3.23e-07, 7.43e-07, 4.15e-07, 1.64e-07, 1.76e-07] [6.34e-07, 6.68e-07, 3.76e-07, 3.23e-07, 7.43e-07, 4.15e-07, 1.64e-07, 1.76e-07] [] 53000 [7.10e-07, 1.24e-06, 3.52e-07, 3.05e-07, 7.05e-07, 3.90e-07, 1.60e-07, 1.55e-07] [6.08e-07, 6.60e-07, 3.52e-07, 3.05e-07, 7.05e-07, 3.90e-07, 1.60e-07, 1.55e-07] [] 54000 [6.92e-07, 1.20e-06, 3.58e-07, 2.99e-07, 6.90e-07, 3.68e-07, 1.96e-07, 1.80e-07] [5.93e-07, 6.46e-07, 3.58e-07, 2.99e-07, 6.90e-07, 3.68e-07, 1.96e-07, 1.80e-07] [] 55000 [6.57e-07, 1.17e-06, 3.28e-07, 4.62e-07, 6.71e-07, 4.29e-07, 1.67e-07, 2.22e-07] [5.60e-07, 6.40e-07, 3.28e-07, 4.62e-07, 6.71e-07, 4.29e-07, 1.67e-07, 2.22e-07] [] 56000 [6.31e-07, 1.13e-06, 3.11e-07, 2.69e-07, 6.25e-07, 3.39e-07, 1.31e-07, 1.42e-07] [5.36e-07, 6.16e-07, 3.11e-07, 2.69e-07, 6.25e-07, 3.39e-07, 1.31e-07, 1.42e-07] [] 57000 [6.34e-07, 1.12e-06, 1.77e-06, 1.87e-06, 8.81e-07, 4.96e-07, 5.62e-07, 4.70e-07] [5.38e-07, 6.54e-07, 1.77e-06, 1.87e-06, 8.81e-07, 4.96e-07, 5.62e-07, 4.70e-07] [] 58000 [6.04e-07, 1.05e-06, 7.01e-07, 1.56e-06, 5.94e-07, 6.18e-07, 2.82e-07, 7.70e-07] [5.10e-07, 5.84e-07, 7.01e-07, 1.56e-06, 5.94e-07, 6.18e-07, 2.82e-07, 7.70e-07] [] 59000 [5.66e-07, 1.03e-06, 2.76e-07, 2.43e-07, 5.61e-07, 3.03e-07, 1.10e-07, 1.30e-07] [4.75e-07, 5.77e-07, 2.76e-07, 2.43e-07, 5.61e-07, 3.03e-07, 1.10e-07, 1.30e-07] [] 60000 [8.51e-07, 1.44e-06, 3.79e-05, 2.66e-05, 1.20e-05, 1.57e-06, 1.73e-05, 4.92e-06] [6.92e-07, 9.58e-07, 3.79e-05, 2.66e-05, 1.20e-05, 1.57e-06, 1.73e-05, 4.92e-06] [] 61000 [5.31e-07, 9.68e-07, 2.75e-07, 2.34e-07, 5.27e-07, 2.84e-07, 9.84e-08, 1.25e-07] [4.43e-07, 5.56e-07, 2.75e-07, 2.34e-07, 5.27e-07, 2.84e-07, 9.84e-08, 1.25e-07] [] 62000 [5.13e-07, 9.41e-07, 2.43e-07, 2.25e-07, 5.18e-07, 2.84e-07, 1.13e-07, 1.31e-07] [4.24e-07, 5.44e-07, 2.43e-07, 2.25e-07, 5.18e-07, 2.84e-07, 1.13e-07, 1.31e-07] [] 63000 [4.96e-07, 9.13e-07, 2.39e-07, 2.16e-07, 4.89e-07, 2.65e-07, 9.18e-08, 1.21e-07] [4.09e-07, 5.31e-07, 2.39e-07, 2.16e-07, 4.89e-07, 2.65e-07, 9.18e-08, 1.21e-07] [] 64000 [4.81e-07, 8.91e-07, 2.63e-07, 3.30e-07, 4.73e-07, 3.00e-07, 9.06e-08, 1.67e-07] [3.94e-07, 5.26e-07, 2.63e-07, 3.30e-07, 4.73e-07, 3.00e-07, 9.06e-08, 1.67e-07] [] 65000 [4.66e-07, 8.66e-07, 3.31e-07, 2.99e-07, 4.77e-07, 2.66e-07, 1.05e-07, 1.32e-07] [3.81e-07, 5.15e-07, 3.31e-07, 2.99e-07, 4.77e-07, 2.66e-07, 1.05e-07, 1.32e-07] [] 66000 [4.58e-07, 8.39e-07, 3.67e-06, 8.30e-07, 3.31e-06, 1.24e-06, 2.88e-06, 9.21e-07] [3.75e-07, 5.01e-07, 3.67e-06, 8.30e-07, 3.31e-06, 1.24e-06, 2.88e-06, 9.21e-07] [] 67000 [4.40e-07, 8.18e-07, 2.17e-07, 1.94e-07, 4.27e-07, 2.40e-07, 7.32e-08, 1.13e-07] [3.55e-07, 4.92e-07, 2.17e-07, 1.94e-07, 4.27e-07, 2.40e-07, 7.32e-08, 1.13e-07] [] 68000 [4.91e-07, 9.10e-07, 6.30e-06, 6.60e-06, 1.73e-06, 8.26e-07, 2.36e-06, 1.54e-06] [3.89e-07, 5.82e-07, 6.30e-06, 6.60e-06, 1.73e-06, 8.26e-07, 2.36e-06, 1.54e-06] [] 69000 [4.23e-07, 7.89e-07, 9.03e-06, 9.35e-06, 1.20e-05, 9.10e-06, 1.02e-05, 8.73e-06] [3.43e-07, 4.66e-07, 9.03e-06, 9.35e-06, 1.20e-05, 9.10e-06, 1.02e-05, 8.73e-06] [] 70000 [5.00e-07, 9.23e-07, 7.05e-06, 1.03e-05, 1.49e-06, 1.57e-06, 2.25e-06, 2.76e-06] [3.86e-07, 6.00e-07, 7.05e-06, 1.03e-05, 1.49e-06, 1.57e-06, 2.25e-06, 2.76e-06] [] 71000 [3.93e-07, 7.34e-07, 1.93e-07, 1.74e-07, 3.77e-07, 2.14e-07, 6.99e-08, 1.08e-07] [3.11e-07, 4.51e-07, 1.93e-07, 1.74e-07, 3.77e-07, 2.14e-07, 6.99e-08, 1.08e-07] [] 72000 [3.82e-07, 7.16e-07, 1.81e-07, 1.59e-07, 3.63e-07, 2.10e-07, 5.98e-08, 1.03e-07] [3.01e-07, 4.43e-07, 1.81e-07, 1.59e-07, 3.63e-07, 2.10e-07, 5.98e-08, 1.03e-07] [] 73000 [3.74e-07, 6.98e-07, 4.65e-07, 2.56e-07, 6.96e-07, 3.80e-07, 4.16e-07, 2.73e-07] [2.91e-07, 4.39e-07, 4.65e-07, 2.56e-07, 6.96e-07, 3.80e-07, 4.16e-07, 2.73e-07] [] 74000 [3.89e-07, 6.96e-07, 1.78e-06, 1.41e-06, 8.35e-07, 2.59e-07, 9.08e-07, 4.09e-07] [3.01e-07, 4.29e-07, 1.78e-06, 1.41e-06, 8.35e-07, 2.59e-07, 9.08e-07, 4.09e-07] [] 75000 [3.68e-07, 6.71e-07, 4.43e-07, 1.01e-06, 3.35e-07, 3.40e-07, 1.87e-07, 5.91e-07] [2.88e-07, 4.33e-07, 4.43e-07, 1.01e-06, 3.35e-07, 3.40e-07, 1.87e-07, 5.91e-07] [] 76000 [3.46e-07, 6.44e-07, 1.64e-07, 1.41e-07, 3.21e-07, 1.93e-07, 5.25e-08, 9.58e-08] [2.67e-07, 4.07e-07, 1.64e-07, 1.41e-07, 3.21e-07, 1.93e-07, 5.25e-08, 9.58e-08] [] 77000 [3.38e-07, 6.28e-07, 1.58e-07, 1.35e-07, 3.11e-07, 1.89e-07, 5.20e-08, 9.41e-08] [2.59e-07, 3.99e-07, 1.58e-07, 1.35e-07, 3.11e-07, 1.89e-07, 5.20e-08, 9.41e-08] [] 78000 [3.31e-07, 6.15e-07, 1.88e-07, 3.18e-07, 3.94e-07, 3.20e-07, 1.31e-07, 2.42e-07] [2.51e-07, 3.95e-07, 1.88e-07, 3.18e-07, 3.94e-07, 3.20e-07, 1.31e-07, 2.42e-07] [] 79000 [3.23e-07, 5.96e-07, 1.52e-07, 1.28e-07, 2.89e-07, 1.83e-07, 5.04e-08, 9.34e-08] [2.47e-07, 3.84e-07, 1.52e-07, 1.28e-07, 2.89e-07, 1.83e-07, 5.04e-08, 9.34e-08] [] 80000 [3.16e-07, 5.77e-07, 1.57e-07, 1.36e-07, 2.86e-07, 1.78e-07, 5.56e-08, 9.30e-08] [2.40e-07, 3.74e-07, 1.57e-07, 1.36e-07, 2.86e-07, 1.78e-07, 5.56e-08, 9.30e-08] [] Best model at step 80000: train loss: 1.80e-06 test loss: 1.52e-06 test metric: [] 'train' took 1653.821573 s Saving loss history to /home/zimbropa/ML-for-PhaseField/loss.dat ... Saving training data to /home/zimbropa/ML-for-PhaseField/train.dat ... Saving test data to /home/zimbropa/ML-for-PhaseField/test.dat ... ╭─    ~/ML-for-PhaseField  on   pzimbrod/issue1 !1 ?3 ·········································································································································· ✔  took 27m 40s   pinn   with zimbropa@pi143  at 11:41:54  ─╮ ╰─ ```

So the entire run took just under 30 Min.

GPU load looked like this:

nvidia-smi output

```bash ╭─    ~ ············································································································································································································ ✔  base   with zimbropa@pi143  at 11:22:35  ─╮ ╰─ nvidia-smi ─╯ Thu Sep 16 11:22:44 2021 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 465.19.01 Driver Version: 465.19.01 CUDA Version: 11.3 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA RTX A5000 On | 00000000:65:00.0 Off | Off | | 59% 84C P2 212W / 230W | 1875MiB / 24256MiB | 58% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 1307 G /usr/lib/xorg/Xorg 10MiB | | 0 N/A N/A 1642 G /usr/bin/gnome-shell 4MiB | | 0 N/A N/A 173044 C python 1857MiB | +-----------------------------------------------------------------------------+ ```

Mem use was 1.86 GB, so rather light. It is of course a simple 1D problem. The GPU has 24 GB at disposal, so i assume that we will be able to work with that.

Running the exact same routine on the CPU instead (Intel Xeon W-2295, 18C @ 3.00 GHz) produces this:

CPU Training

```bash ╭─    ~/ML-for-PhaseField  on   pzimbrod/issue1 !1 ?3 ·········································································································································· ✔  took 27m 40s   pinn   with zimbropa@pi143  at 11:41:54  ─╮ ╰─ DDEBACKEND=pytorch CUDA_VISIBLE_DEVICES="" python models/diffusion_1d_inverse.py ─╯ Using backend: pytorch Warning: 50000 points required, but 50055 points sampled. Compiling model... 'compile' took 0.000087 s Training model... Step Train loss Test loss Test metric 0 [3.38e-02, 4.00e-02, 3.78e-01, 3.32e-01, 3.18e-02, 3.22e-02, 6.12e-01, 4.95e-01] [3.31e-02, 3.89e-02, 31000 [1.80e-04, 9.70e-05, 5.80e-04, 4.71e-04, 1.18e-03, 9.19e-04, 2.48e-04, 2.02e-04] [1.49e-04, 7.53e-05, 5.80e-04, 4.71e-04, 1.18e-03, 9.19e-04, 2.48e-04, 2.02e-04] [] 2000 [4.26e-05, 4.32e-05, 1.91e-04, 1.12e-04, 3.94e-04, 2.40e-04, 7.09e-05, 9.47e-05] [3.35e-05, 3.78e-05, 1.91e-04, 1.12e-04, 3.94e-04, 2.40e-04, 7.09e-05, 9.47e-05] [] 3000 [3.38e-05, 5.86e-05, 6.65e-05, 2.73e-05, 1.43e-04, 3.57e-05, 2.78e-05, 6.37e-05] [3.06e-05, 5.30e-05, 6.65e-05, 2.73e-05, 1.43e-04, 3.57e-05, 2.78e-05, 6.37e-05] [] 4000 [3.82e-05, 5.81e-05, 3.06e-05, 1.47e-05, 5.41e-05, 9.68e-06, 2.13e-05, 5.15e-05] [3.51e-05, 5.11e-05, 3.06e-05, 1.47e-05, 5.41e-05, 9.68e-06, 2.13e-05, 5.15e-05] [] 5000 [3.50e-05, 5.12e-05, 3.37e-05, 3.46e-05, 3.06e-05, 2.05e-05, 1.99e-05, 7.54e-05] [3.18e-05, 4.35e-05, 3.37e-05, 3.46e-05, 3.06e-05, 2.05e-05, 1.99e-05, 7.54e-05] [] 6000 [2.72e-05, 4.53e-05, 1.41e-05, 8.34e-06, 1.88e-05, 6.49e-06, 1.07e-05, 3.57e-05] [2.41e-05, 3.84e-05, 1.41e-05, 8.34e-06, 1.88e-05, 6.49e-06, 1.07e-05, 3.57e-05] [] 7000 [2.13e-05, 4.16e-05, 1.16e-05, 6.87e-06, 1.52e-05, 6.02e-06, 8.64e-06, 2.92e-05] [1.86e-05, 3.47e-05, 1.16e-05, 6.87e-06, 1.52e-05, 6.02e-06, 8.64e-06, 2.92e-05] [] 8000 [1.72e-05, 3.85e-05, 9.83e-06, 5.66e-06, 1.32e-05, 5.39e-06, 7.37e-06, 2.33e-05] [1.48e-05, 3.15e-05, 9.83e-06, 5.66e-06, 1.32e-05, 5.39e-06, 7.37e-06, 2.33e-05] [] 9000 [1.44e-05, 3.53e-05, 8.41e-06, 4.78e-06, 1.17e-05, 4.71e-06, 6.12e-06, 1.80e-05] [1.25e-05, 2.80e-05, 8.41e-06, 4.78e-06, 1.17e-05, 4.71e-06, 6.12e-06, 1.80e-05] [] 10000 [1.28e-05, 3.18e-05, 7.16e-06, 4.05e-06, 1.04e-05, 4.10e-06, 5.08e-06, 1.38e-05] [1.13e-05, 2.42e-05, 7.16e-06, 4.05e-06, 1.04e-05, 4.10e-06, 5.08e-06, 1.38e-05] [] 11000 [1.22e-05, 2.86e-05, 6.08e-06, 3.17e-06, 9.45e-06, 3.71e-06, 4.43e-06, 1.13e-05] [1.10e-05, 2.06e-05, 6.08e-06, 3.17e-06, 9.45e-06, 3.71e-06, 4.43e-06, 1.13e-05] [] 12000 [1.18e-05, 2.56e-05, 5.27e-06, 3.15e-06, 8.54e-06, 3.11e-06, 3.96e-06, 8.82e-06] [1.09e-05, 1.79e-05, 5.27e-06, 3.15e-06, 8.54e-06, 3.11e-06, 3.96e-06, 8.82e-06] [] 13000 [1.13e-05, 2.19e-05, 3.12e-05, 7.37e-05, 8.65e-06, 2.53e-05, 1.10e-05, 3.52e-05] [1.01e-05, 1.64e-05, 3.12e-05, 7.37e-05, 8.65e-06, 2.53e-05, 1.10e-05, 3.52e-05] [] 14000 [1.07e-05, 2.10e-05, 3.93e-06, 2.54e-06, 7.39e-06, 2.51e-06, 3.40e-06, 6.51e-06] [9.92e-06, 1.40e-05, 3.93e-06, 2.54e-06, 7.39e-06, 2.51e-06, 3.40e-06, 6.51e-06] [] 15000 [1.00e-05, 1.89e-05, 3.42e-06, 2.39e-06, 6.96e-06, 2.32e-06, 3.21e-06, 5.65e-06] [9.19e-06, 1.24e-05, 3.42e-06, 2.39e-06, 6.96e-06, 2.32e-06, 3.21e-06, 5.65e-06] [] 16000 [9.62e-06, 1.60e-05, 4.42e-05, 1.20e-04, 7.51e-06, 4.67e-05, 9.91e-06, 5.54e-05] [8.47e-06, 1.20e-05, 4.42e-05, 1.20e-04, 7.51e-06, 4.67e-05, 9.91e-06, 5.54e-05] [] 17000 [8.38e-06, 1.49e-05, 2.68e-06, 2.09e-06, 6.28e-06, 2.12e-06, 2.67e-06, 4.38e-06] [7.60e-06, 9.62e-06, 2.68e-06, 2.09e-06, 6.28e-06, 2.12e-06, 2.67e-06, 4.38e-06] [] 18000 [7.61e-06, 1.32e-05, 2.35e-06, 1.91e-06, 5.95e-06, 2.07e-06, 2.33e-06, 3.85e-06] [6.83e-06, 8.41e-06, 2.35e-06, 1.91e-06, 5.95e-06, 2.07e-06, 2.33e-06, 3.85e-06] [] 19000 [6.91e-06, 1.14e-05, 2.12e-06, 1.88e-06, 5.49e-06, 2.02e-06, 2.08e-06, 3.38e-06] [6.11e-06, 7.29e-06, 2.12e-06, 1.88e-06, 5.49e-06, 2.02e-06, 2.08e-06, 3.38e-06] [] 20000 [6.28e-06, 9.97e-06, 1.77e-06, 1.61e-06, 5.10e-06, 1.95e-06, 1.77e-06, 3.01e-06] [5.45e-06, 6.33e-06, 1.77e-06, 1.61e-06, 5.10e-06, 1.95e-06, 1.77e-06, 3.01e-06] [] 21000 [5.57e-06, 8.16e-06, 3.99e-06, 5.27e-06, 4.83e-06, 2.59e-06, 2.57e-06, 3.62e-06] [4.58e-06, 5.69e-06, 3.99e-06, 5.27e-06, 4.83e-06, 2.59e-06, 2.57e-06, 3.62e-06] [] 22000 [5.15e-06, 7.80e-06, 1.42e-06, 1.40e-06, 4.26e-06, 1.76e-06, 1.31e-06, 2.19e-06] [4.23e-06, 5.01e-06, 1.42e-06, 1.40e-06, 4.26e-06, 1.76e-06, 1.31e-06, 2.19e-06] [] 23000 [4.63e-06, 6.80e-06, 1.73e-06, 1.56e-06, 4.30e-06, 1.62e-06, 1.64e-06, 2.31e-06] [3.66e-06, 4.36e-06, 1.73e-06, 1.56e-06, 4.30e-06, 1.62e-06, 1.64e-06, 2.31e-06] [] 24000 [4.11e-06, 6.18e-06, 1.22e-06, 1.23e-06, 3.61e-06, 1.56e-06, 9.62e-07, 1.60e-06] [3.16e-06, 4.02e-06, 1.22e-06, 1.23e-06, 3.61e-06, 1.56e-06, 9.62e-07, 1.60e-06] [] 25000 [3.66e-06, 5.39e-06, 2.05e-06, 2.09e-06, 3.90e-06, 1.49e-06, 1.58e-06, 2.10e-06] [2.70e-06, 3.52e-06, 2.05e-06, 2.09e-06, 3.90e-06, 1.49e-06, 1.58e-06, 2.10e-06] [] 26000 [5.36e-06, 8.62e-06, 1.33e-04, 2.95e-04, 1.75e-05, 1.08e-04, 5.51e-05, 1.83e-04] [3.98e-06, 4.42e-06, 1.33e-04, 2.95e-04, 1.75e-05, 1.08e-04, 5.51e-05, 1.83e-04] [] 27000 [2.96e-06, 4.13e-06, 3.75e-06, 3.11e-06, 5.54e-06, 4.11e-06, 2.44e-06, 6.26e-06] [2.08e-06, 2.89e-06, 3.75e-06, 3.11e-06, 5.54e-06, 4.11e-06, 2.44e-06, 6.26e-06] [] 28000 [2.56e-06, 3.90e-06, 9.40e-07, 9.89e-07, 2.70e-06, 1.28e-06, 4.63e-07, 9.07e-07] [1.70e-06, 2.58e-06, 9.40e-07, 9.89e-07, 2.70e-06, 1.28e-06, 4.63e-07, 9.07e-07] [] 29000 [2.30e-06, 3.49e-06, 8.90e-07, 9.45e-07, 2.51e-06, 1.21e-06, 3.79e-07, 8.08e-07] [1.48e-06, 2.32e-06, 8.90e-07, 9.45e-07, 2.51e-06, 1.21e-06, 3.79e-07, 8.08e-07] [] 30000 [2.10e-06, 3.06e-06, 1.90e-06, 1.76e-06, 3.13e-06, 1.15e-06, 1.50e-06, 1.43e-06] [1.30e-06, 2.03e-06, 1.90e-06, 1.76e-06, 3.13e-06, 1.15e-06, 1.50e-06, 1.43e-06] [] 31000 [1.92e-06, 2.88e-06, 7.94e-07, 8.47e-07, 2.18e-06, 1.08e-06, 2.53e-07, 6.53e-07] [1.15e-06, 1.91e-06, 7.94e-07, 8.47e-07, 2.18e-06, 1.08e-06, 2.53e-07, 6.53e-07] [] 32000 [1.71e-06, 2.49e-06, 2.37e-06, 1.41e-06, 3.12e-06, 1.06e-06, 1.79e-06, 1.32e-06] [9.83e-07, 1.70e-06, 2.37e-06, 1.41e-06, 3.12e-06, 1.06e-06, 1.79e-06, 1.32e-06] [] 33000 [1.67e-06, 2.43e-06, 7.31e-07, 7.68e-07, 1.91e-06, 9.59e-07, 2.07e-07, 5.64e-07] [9.69e-07, 1.62e-06, 7.31e-07, 7.68e-07, 1.91e-06, 9.59e-07, 2.07e-07, 5.64e-07] [] 34000 [2.78e-06, 4.29e-06, 9.10e-05, 1.56e-04, 1.60e-05, 4.52e-05, 3.90e-05, 8.00e-05] [1.70e-06, 2.41e-06, 9.10e-05, 1.56e-04, 1.60e-05, 4.52e-05, 3.90e-05, 8.00e-05] [] 35000 [1.49e-06, 2.15e-06, 6.40e-07, 6.75e-07, 1.68e-06, 8.69e-07, 1.45e-07, 4.44e-07] [8.26e-07, 1.44e-06, 6.40e-07, 6.75e-07, 1.68e-06, 8.69e-07, 1.45e-07, 4.44e-07] [] 36000 [1.42e-06, 2.03e-06, 7.23e-07, 7.92e-07, 1.62e-06, 8.37e-07, 1.85e-07, 5.09e-07] [7.89e-07, 1.37e-06, 7.23e-07, 7.92e-07, 1.62e-06, 8.37e-07, 1.85e-07, 5.09e-07] [] 37000 [1.36e-06, 1.95e-06, 5.78e-07, 6.02e-07, 1.51e-06, 7.89e-07, 1.32e-07, 3.77e-07] [7.46e-07, 1.33e-06, 5.78e-07, 6.02e-07, 1.51e-06, 7.89e-07, 1.32e-07, 3.77e-07] [] 38000 [1.30e-06, 1.87e-06, 5.56e-07, 5.74e-07, 1.44e-06, 7.54e-07, 1.30e-07, 3.54e-07] [7.14e-07, 1.28e-06, 5.56e-07, 5.74e-07, 1.44e-06, 7.54e-07, 1.30e-07, 3.54e-07] [] 39000 [1.26e-06, 1.80e-06, 3.85e-06, 1.74e-06, 3.07e-06, 9.88e-07, 2.47e-06, 5.70e-07] [6.94e-07, 1.28e-06, 3.85e-06, 1.74e-06, 3.07e-06, 9.88e-07, 2.47e-06, 5.70e-07] [] 40000 [1.59e-06, 2.55e-06, 2.70e-05, 6.26e-05, 4.45e-06, 1.96e-05, 1.09e-05, 2.91e-05] [1.01e-06, 1.92e-06, 2.70e-05, 6.26e-05, 4.45e-06, 1.96e-05, 1.09e-05, 2.91e-05] [] 41000 [1.14e-06, 1.65e-06, 6.67e-07, 1.08e-06, 1.25e-06, 9.68e-07, 1.95e-07, 6.81e-07] [6.50e-07, 1.19e-06, 6.67e-07, 1.08e-06, 1.25e-06, 9.68e-07, 1.95e-07, 6.81e-07] [] 42000 [1.11e-06, 1.61e-06, 4.73e-07, 4.68e-07, 1.20e-06, 6.44e-07, 1.30e-07, 2.82e-07] [6.09e-07, 1.12e-06, 4.73e-07, 4.68e-07, 1.20e-06, 6.44e-07, 1.30e-07, 2.82e-07] [] 43000 [1.07e-06, 1.55e-06, 4.56e-07, 4.47e-07, 1.15e-06, 6.20e-07, 1.31e-07, 2.70e-07] [5.86e-07, 1.09e-06, 4.56e-07, 4.47e-07, 1.15e-06, 6.20e-07, 1.31e-07, 2.70e-07] [] 44000 [1.02e-06, 1.47e-06, 6.29e-07, 5.10e-07, 1.26e-06, 6.35e-07, 3.42e-07, 3.04e-07] [5.68e-07, 1.07e-06, 6.29e-07, 5.10e-07, 1.26e-06, 6.35e-07, 3.42e-07, 3.04e-07] [] 45000 [9.89e-07, 1.44e-06, 4.24e-07, 4.05e-07, 1.06e-06, 5.77e-07, 1.32e-07, 2.44e-07] [5.37e-07, 1.02e-06, 4.24e-07, 4.05e-07, 1.06e-06, 5.77e-07, 1.32e-07, 2.44e-07] [] 46000 [9.51e-07, 1.39e-06, 4.09e-07, 3.88e-07, 1.02e-06, 5.58e-07, 1.33e-07, 2.35e-07] [5.17e-07, 9.87e-07, 4.09e-07, 3.88e-07, 1.02e-06, 5.58e-07, 1.33e-07, 2.35e-07] [] 47000 [9.14e-07, 1.34e-06, 3.96e-07, 3.71e-07, 9.84e-07, 5.39e-07, 1.33e-07, 2.23e-07] [4.95e-07, 9.53e-07, 3.96e-07, 3.71e-07, 9.84e-07, 5.39e-07, 1.33e-07, 2.23e-07] [] 48000 [8.78e-07, 1.29e-06, 3.83e-07, 3.56e-07, 9.49e-07, 5.22e-07, 1.34e-07, 2.16e-07] [4.76e-07, 9.25e-07, 3.83e-07, 3.56e-07, 9.49e-07, 5.22e-07, 1.34e-07, 2.16e-07] [] 49000 [8.43e-07, 1.25e-06, 3.72e-07, 3.50e-07, 9.14e-07, 5.05e-07, 1.41e-07, 2.13e-07] [4.63e-07, 8.98e-07, 3.72e-07, 3.50e-07, 9.14e-07, 5.05e-07, 1.41e-07, 2.13e-07] [] 50000 [8.08e-07, 1.20e-06, 5.35e-07, 7.15e-07, 1.12e-06, 8.43e-07, 4.47e-07, 5.69e-07] [4.44e-07, 8.75e-07, 5.35e-07, 7.15e-07, 1.12e-06, 8.43e-07, 4.47e-07, 5.69e-07] [] 51000 [1.18e-06, 1.90e-06, 7.78e-05, 1.37e-04, 1.40e-05, 3.14e-05, 2.84e-05, 5.24e-05] [7.34e-07, 1.41e-06, 7.78e-05, 1.37e-04, 1.40e-05, 3.14e-05, 2.84e-05, 5.24e-05] [] 52000 [7.59e-07, 1.13e-06, 3.35e-07, 3.06e-07, 8.31e-07, 4.63e-07, 1.36e-07, 1.86e-07] [4.14e-07, 8.18e-07, 3.35e-07, 3.06e-07, 8.31e-07, 4.63e-07, 1.36e-07, 1.86e-07] [] 53000 [7.32e-07, 1.09e-06, 3.24e-07, 2.96e-07, 8.06e-07, 4.50e-07, 1.37e-07, 1.80e-07] [4.00e-07, 7.90e-07, 3.24e-07, 2.96e-07, 8.06e-07, 4.50e-07, 1.37e-07, 1.80e-07] [] 54000 [7.09e-07, 1.06e-06, 3.16e-07, 2.90e-07, 7.80e-07, 4.37e-07, 1.38e-07, 1.79e-07] [3.93e-07, 7.71e-07, 3.16e-07, 2.90e-07, 7.80e-07, 4.37e-07, 1.38e-07, 1.79e-07] [] 55000 [6.86e-07, 1.03e-06, 3.15e-07, 2.84e-07, 7.58e-07, 4.26e-07, 1.38e-07, 1.78e-07] [3.84e-07, 7.50e-07, 3.15e-07, 2.84e-07, 7.58e-07, 4.26e-07, 1.38e-07, 1.78e-07] [] 56000 [6.68e-07, 9.92e-07, 4.29e-07, 3.26e-07, 8.38e-07, 4.44e-07, 3.01e-07, 1.67e-07] [3.68e-07, 7.24e-07, 4.29e-07, 3.26e-07, 8.38e-07, 4.44e-07, 3.01e-07, 1.67e-07] [] 57000 [6.48e-07, 9.56e-07, 2.85e-07, 2.61e-07, 7.16e-07, 4.07e-07, 1.36e-07, 1.55e-07] [3.61e-07, 6.98e-07, 2.85e-07, 2.61e-07, 7.16e-07, 4.07e-07, 1.36e-07, 1.55e-07] [] 58000 [6.31e-07, 9.25e-07, 2.75e-07, 2.54e-07, 6.97e-07, 3.99e-07, 1.36e-07, 1.49e-07] [3.54e-07, 6.78e-07, 2.75e-07, 2.54e-07, 6.97e-07, 3.99e-07, 1.36e-07, 1.49e-07] [] 59000 [6.16e-07, 8.96e-07, 2.66e-07, 2.47e-07, 6.77e-07, 3.90e-07, 1.37e-07, 1.45e-07] [3.48e-07, 6.60e-07, 2.66e-07, 2.47e-07, 6.77e-07, 3.90e-07, 1.37e-07, 1.45e-07] [] 60000 [6.00e-07, 8.68e-07, 2.58e-07, 2.43e-07, 6.57e-07, 3.82e-07, 1.38e-07, 1.43e-07] [3.44e-07, 6.43e-07, 2.58e-07, 2.43e-07, 6.57e-07, 3.82e-07, 1.38e-07, 1.43e-07] [] 61000 [5.78e-07, 8.49e-07, 6.17e-07, 3.44e-07, 7.54e-07, 4.12e-07, 4.08e-07, 2.69e-07] [3.41e-07, 6.55e-07, 6.17e-07, 3.44e-07, 7.54e-07, 4.12e-07, 4.08e-07, 2.69e-07] [] 62000 [5.73e-07, 8.09e-07, 2.43e-07, 2.34e-07, 6.25e-07, 3.67e-07, 1.37e-07, 1.33e-07] [3.32e-07, 6.02e-07, 2.43e-07, 2.34e-07, 6.25e-07, 3.67e-07, 1.37e-07, 1.33e-07] [] 63000 [5.61e-07, 7.85e-07, 2.34e-07, 2.27e-07, 6.08e-07, 3.60e-07, 1.37e-07, 1.27e-07] [3.29e-07, 5.87e-07, 2.34e-07, 2.27e-07, 6.08e-07, 3.60e-07, 1.37e-07, 1.27e-07] [] 64000 [5.66e-07, 7.79e-07, 3.59e-06, 1.30e-05, 6.01e-06, 1.27e-05, 5.12e-06, 1.36e-05] [3.25e-07, 5.81e-07, 3.59e-06, 1.30e-05, 6.01e-06, 1.27e-05, 5.12e-06, 1.36e-05] [] 65000 [5.38e-07, 7.33e-07, 2.20e-07, 2.18e-07, 5.77e-07, 3.47e-07, 1.37e-07, 1.19e-07] [3.21e-07, 5.51e-07, 2.20e-07, 2.18e-07, 5.77e-07, 3.47e-07, 1.37e-07, 1.19e-07] [] 66000 [5.29e-07, 7.15e-07, 2.18e-07, 2.16e-07, 5.62e-07, 3.41e-07, 1.36e-07, 1.18e-07] [3.20e-07, 5.40e-07, 2.18e-07, 2.16e-07, 5.62e-07, 3.41e-07, 1.36e-07, 1.18e-07] [] 67000 [5.18e-07, 6.91e-07, 2.54e-07, 3.57e-07, 5.52e-07, 3.81e-07, 1.47e-07, 1.91e-07] [3.14e-07, 5.21e-07, 2.54e-07, 3.57e-07, 5.52e-07, 3.81e-07, 1.47e-07, 1.91e-07] [] 68000 [5.11e-07, 6.73e-07, 3.74e-07, 5.74e-07, 5.56e-07, 4.27e-07, 1.92e-07, 3.00e-07] [3.10e-07, 5.10e-07, 3.74e-07, 5.74e-07, 5.56e-07, 4.27e-07, 1.92e-07, 3.00e-07] [] 69000 [5.34e-07, 7.09e-07, 3.38e-06, 6.37e-06, 9.70e-07, 2.01e-06, 1.24e-06, 2.99e-06] [3.19e-07, 5.33e-07, 3.38e-06, 6.37e-06, 9.70e-07, 2.01e-06, 1.24e-06, 2.99e-06] [] 70000 [5.27e-07, 6.72e-07, 4.88e-06, 5.65e-06, 1.84e-06, 1.24e-06, 2.33e-06, 1.55e-06] [3.54e-07, 5.21e-07, 4.88e-06, 5.65e-06, 1.84e-06, 1.24e-06, 2.33e-06, 1.55e-06] [] 71000 [5.42e-07, 7.14e-07, 6.04e-06, 9.97e-06, 1.38e-06, 2.47e-06, 2.26e-06, 4.19e-06] [3.26e-07, 5.37e-07, 6.04e-06, 9.97e-06, 1.38e-06, 2.47e-06, 2.26e-06, 4.19e-06] [] 72000 [4.77e-07, 6.08e-07, 2.80e-07, 4.02e-07, 5.12e-07, 3.83e-07, 1.58e-07, 2.10e-07] [3.02e-07, 4.69e-07, 2.80e-07, 4.02e-07, 5.12e-07, 3.83e-07, 1.58e-07, 2.10e-07] [] 73000 [4.71e-07, 5.79e-07, 1.48e-06, 1.18e-06, 1.83e-06, 1.73e-06, 1.38e-06, 1.58e-06] [3.00e-07, 4.44e-07, 1.48e-06, 1.18e-06, 1.83e-06, 1.73e-06, 1.38e-06, 1.58e-06] [] 74000 [4.70e-07, 5.69e-07, 6.41e-07, 3.56e-07, 7.98e-07, 4.70e-07, 5.70e-07, 2.46e-07] [3.05e-07, 4.38e-07, 6.41e-07, 3.56e-07, 7.98e-07, 4.70e-07, 5.70e-07, 2.46e-07] [] 75000 [4.62e-07, 5.52e-07, 2.27e-07, 2.98e-07, 4.62e-07, 3.27e-07, 1.57e-07, 1.42e-07] [3.02e-07, 4.26e-07, 2.27e-07, 2.98e-07, 4.62e-07, 3.27e-07, 1.57e-07, 1.42e-07] [] 76000 [4.59e-07, 5.42e-07, 2.09e-07, 2.54e-07, 4.53e-07, 3.03e-07, 1.42e-07, 1.31e-07] [2.99e-07, 4.19e-07, 2.09e-07, 2.54e-07, 4.53e-07, 3.03e-07, 1.42e-07, 1.31e-07] [] 77000 [5.78e-07, 7.36e-07, 1.02e-05, 2.07e-05, 1.83e-06, 6.54e-06, 4.18e-06, 1.12e-05] [3.81e-07, 5.83e-07, 1.02e-05, 2.07e-05, 1.83e-06, 6.54e-06, 4.18e-06, 1.12e-05] [] 78000 [4.48e-07, 5.12e-07, 1.71e-07, 1.92e-07, 4.32e-07, 2.83e-07, 1.30e-07, 9.57e-08] [2.96e-07, 3.98e-07, 1.71e-07, 1.92e-07, 4.32e-07, 2.83e-07, 1.30e-07, 9.57e-08] [] 79000 [4.38e-07, 5.00e-07, 1.80e-06, 4.97e-06, 2.72e-06, 5.29e-06, 2.02e-06, 5.74e-06] [2.96e-07, 3.87e-07, 1.80e-06, 4.97e-06, 2.72e-06, 5.29e-06, 2.02e-06, 5.74e-06] [] 80000 [4.40e-07, 4.95e-07, 2.07e-07, 2.52e-07, 4.26e-07, 2.91e-07, 1.41e-07, 1.29e-07] [2.94e-07, 3.85e-07, 2.07e-07, 2.52e-07, 4.26e-07, 2.91e-07, 1.41e-07, 1.29e-07] [] Best model at step 78000: train loss: 2.26e-06 test loss: 2.00e-06 test metric: [] 'train' took 11220.048939 s Saving loss history to /home/zimbropa/ML-for-PhaseField/loss.dat ... Saving training data to /home/zimbropa/ML-for-PhaseField/train.dat ... Saving test data to /home/zimbropa/ML-for-PhaseField/test.dat ... ╭─    ~/ML-for-PhaseField  on   pzimbrod/issue1 !1 ?3 ········································································································································· ✔  took 3h 7m 2s   pinn   with zimbropa@pi143 ```

As you can see, training was considerably (almost 7x) slower to perform than on GPU for the same architecture, dataset and training strategy.

Mem use according to htop was around 1.7 GB peak. CPU use was 100 % on all cores, though only half the threads were utilized. I suspect this behaviour is deliberate since e.g. when running OpenFOAM via MPI the same strategy is recommended. Probably scheduling and processing two tasks on the same core in parallel does not overweigh the use of idle time and creates unnecessary I/O.

Convergence behavior is fairly similar I'd say. I wouldn't compare the residuals per epoch directly as we used the ADAM optimizer, which utilizes stochastic gradient descent.

I'm quite excited to find out how time-to-solution scales with growing dimensions, training set size and model complexity, i.e. the nature of the physics we model.

For inverse problems, I'd say that that solution time is less important than accuracy as there is little hope to find an accurate solution in reasonable time otherwise. For forward problems though, I'd be very interested in comparing overall time to an established numerical algorithm using a comparable grid. Maybe that's something to look into for our coupled phase field problem as well.

[pzimbrod]

Closing, as we've settled on this problem to work with.