search

meom-group / eophis

Couple Python and Machine Learning models with Geophysical simulation codes through OASIS
https://eophis.readthedocs.io/en/latest/
MIT License
9 stars 1 forks source link

Running inference models on multiple CPUs ? #1

Closed lesommer closed 4 months ago

lesommer commented 11 months ago

Eophis allows to define python models that can be coupled to timestepping codes (as NEMO) through OASIS. But as present, the python runs on a single processor.

Eophis should probably eventually allow to define python models that run on different processors. But it is not clear yet how Eophis should deal with inference on multiple CPUs.

A a priori design requirement would be to keep the users-provided python code as agnostic as possible to the hardware constraints.

alexis-barge commented 4 months ago

Eophis_v1.0.0 has new features to run Python models on grid scattered over several processes with automatically reconstructed overlapping cells, allowing to use CNNs and differential calculus on partial subdomain.

User only has to define global grid with number of desired halos and choose number of executing processes.

Successful NEMO experiment with exchange of whole 3D grids for gradient-based model.