Closed franzpoeschel closed 2 years ago
Solution: Allow to truncate a number of last steps when opening a file for appending
Just to add, more precisely, we would like to specify "truncate all steps after ADIOS step <N>
"
@franzpoeschel BP5 in the master branch now has a parameter AppendAfterSteps. It enables overwriting some steps as you requested. If you have 10 steps in the file,
Default value is MAX_INT, so it always appends after the last step. -1 would be another good default value.
We won't add this feature to BP4 because it would require implementing the entire metadata parsing process of the read side in the writing engine.
Great, thanks for this!
This is finally going to work properly in 2.8.1
Why is this feature important? Scenario:
Consequence: After restarting from a checkpoint:
=> The output file will contain the same steps twice
Solution: Allow to truncate a number of last steps when opening a file for appending
What is the potential impact of this feature in the community? ADIOS steps can be used more sensibly for output in simulations that use checkpointing
Is your feature request related to a problem? Please describe. see above
Describe the solution you'd like and potential required effort see above effort depends on the file format
Describe alternatives you've considered and potential required effort Alternative: Just accept that the output file will have some time steps twice. Required effort: Additional complexity in reading data produced by ADIOS
Additional context Discussed offline with @pnorbert and @ax3l https://github.com/openPMD/openPMD-api/pull/1007