aiidateam / disk-objectstore

An implementation of an efficient "object store" (actually, a key-value store) writing files on disk and not requiring a running server
https://disk-objectstore.readthedocs.io
MIT License
15 stars 8 forks source link

Extend the `export` function to write to loose objects instead of only to packs #79

Closed giovannipizzi closed 4 years ago

giovannipizzi commented 4 years ago

The function has been added in #75

We can add a new parameter to_pack=True by default, in order not to change the default behaviour. In this case we would ignore the compress flag. Note that before implementing this, we need to address #78 as now there is no way to store an object from a stream to avoid excessive memory usage.

COMMENT: we will need if this is needed, otherwise we just close this issue since this can be implentented with a simple loop calling add_object many times and there isn't probably much optimisation that can be done.

giovannipizzi commented 4 years ago

Closing for now because of the comment above - just pinging @sphuber in case this will be needed for AiiDA and a simple loop (in the case of loose objects) is not enough, we can reopen this