mpiraux
commented
1 year ago If we were to execute mpf in a notebook, then outputing numpy or panda dataframes could be just fine. And we leverage the notebook to be able to save these structures, for instance in Pickle files. This alleviates the need for a dedicated storage solution.
To fill missing experiments from a previous run, one could simply pass the results back to mpf.
I would prefer mpf to be explicit about the storage of collected results. For that purpose I would avoid a file format that cannot reasonably self-describe. I expect the magnitude of data collected through mpf to be relatively low in most cases so the description overhead will remain acceptable. Also, each experiment should produce a separate set of results, probably in a separate file. The file should of course be machine readable so I would aim for YAML.