search

openPMD / openPMD-api

:floppy_disk: C++ & Python API for Scientific I/O
https://openpmd-api.readthedocs.io
GNU Lesser General Public License v3.0
142 stars 51 forks source link

Bind to Conduit Blueprints #1330

Open ax3l opened 2 years ago

ax3l commented 2 years ago

It would be interesting to explore a direct connection to Conduit and it's Blueprints for openPMD.

https://llnl-conduit.readthedocs.io

Also, the flexibility of the conduit::Node object could be inspiring for our own C++ object hierarchy.

bussmann commented 2 years ago

I like the idea very much. I have said more than once that essentially everything we do in the code is on data exchange thst could follow an openPMD-style API, as the objects we deal with are particles and meshes. An obvious first candidate is ISAAC to me but there are other parts, especially plugins that could benefit from this. I thus very much like the idea of Conduit and would not be shy to drive it to extremes, because 'why not?'.

franzpoeschel commented 2 years ago

Sounds definitely interesting as a further backend option. Side note: For intra-process data exchange, a somewhat lower-hanging fruit would be the Inline Engine of ADIOS2, which we don't support yet. The greater challenge (and reason why we did not yet really pursue this path) would be that we would need to either (1) add greater flexibility in memory representation to the openPMD standard or (2) change those codes to use a SoA layout as openPMD does so far. (considering recent attempts to use openPMD in the context of ML where each single data point is often already a large vector, I think that a greater memory flexibility in openPMD is justified/needed, cc @RandomDefaultUser)