openPMD: Implement High-Level C++ Interfaces

[ax3l]

https://github.com/ComputationalRadiationPhysics/openPMD

Use the internal file format update to change some minor conventions and to add some more group attributes as they are defined in the openPMD base standard (not the ED-PIC extension).

main tasks:

• [ ] adding the required attributes to / and to each record (units and time)
• [ ] adding read/write support for constant components
• [x] allow to write real global attributes to / #173
• [x] string read/write support #167
• [x] adding HDF5 dataspaces for attributes for unitDimension array of 7 float64 #170 #171
• particle specific:
  • [ ] rename "poly-data" to particle based in documents
  • [ ] ensure the position record is always present
  • [ ] defining the particlePatches record for parallel writes
• mesh specific
  • [ ] rename "field"/"grid" to mesh based in documents
  • [ ] add mesh-specific attributes for each record
  • [ ] add "position" attribute for each mesh record component

for several of these points, a more object oriented API with actual mesh and particle objects might be necessary. Due to that, I propose to upgrade the major version of the library to 2.0 with the changes.

[RemiLehe]

I was wondering: is 2.0 necessary for openPMD, or can openPMD already be implemented in the next release of libSplash (1.3/1.4 as I understood) ?

[ax3l]

@RemiLehe Yes you understood it correctly. One can implement the openPMD markup already with the lastest release of libSplash, version 1.3.0.

For example, PIConGPU is currently implementing openPMD 1.0.0 for HDF5 based on libSplash 1.3.0 (development branch).

The idea of this PR is to define a C++ high-level interface in 2.0.0, so the writeGlobalAttribute / writeAttribute calls to the libSplash API for field and particle records are done transparently in the background. This is just convenience so people can think in openPMD objects rather than low-level write/read calls. Also, constant records can be handled transparently without fiddling with dirs and attributes in if-clauses.

[RemiLehe]

Ok, thanks for the update ! I think it is great to have a high-level interface for this. Btw, could this allow other PIC codes to use libSplash as a library that writes openPMD output, without having to specifically write the underlying HDF5/ADIOS code ? (On the LBL side, we've had several request for this kind of library.)

[ax3l]

Btw, could this allow other PIC codes to use libSplash as a library that writes openPMD output, without having to specifically write the underlying HDF5/ADIOS code ?

Yes, we encourage to do so for HDF5! :)

libSplash was designed from the beginning years ago to be particle and (simple) mesh data centric, using a subset of what HDF5 offers. It evolved over time from parallel file-per-rank to MPI-I/O output methods and can be seen as a predecessor to the openPMD idea. For example, it does not automatically write a large quantity of attributes by itself besides sizes and offsets (but users can add them with simple calls like writeAttribute and writeGlobalAttribute to fulfil the openPMD base standard and it's extensions).

So yes, users of libSplash do not need to know the low-lvl file format (HDF5) APIs, data structures, dataspaces, etc. at all.

Regarding ADIOS writing: in libSplash we first should develop a high-lvl interface for openPMD objects (this issue) and then add an other backend to keep the required changes small.

It's probably possible to add ADIOS as an additional parallel writer, but this will take more planning and time. I will open a separate issue for that: #215

[ax3l]

This issue is super-seeded by our new project, openPMD-api, which will receive all further I/O efforts.