openPMD / openPMD-api

:floppy_disk: C++ & Python API for Scientific I/O
https://openpmd-api.readthedocs.io
GNU Lesser General Public License v3.0
142 stars 51 forks source link

Explicit user-exposed Series close/(re-)open functionality #149

Open C0nsultant opened 6 years ago

C0nsultant commented 6 years ago

Currently, the lifetime of file handles used in the backends is tied to the lifetime of a Series they are contained in. It would be very convenient to offer explicit control over this lifetime by having a public interface to close all handles in a Series (e.g. through Series::close() and likewise to re-open a closed Series (e.g. through Series::reopen()). Sidenote: When reopening a ~file~ series, it would be wise to re-read ~the entire file~ the meta-data of all involved files instead of resuing old values stored in this logical API. The reasoning is that the file could be modified externally, resulting in an incosistent state between logical and physical representation.

ax3l commented 6 years ago

re-read the entire file

yes, meta-data wise - and even the whole series, not only a single file. Also, it would be useful to add an option to explicitly skip this for performance (taking the risk into account the files below changed - if one can control it by other means).

Series::reopen( bool verify = true );