SciQLop / speasy

Space Physics made EASY! A simple Python package to deal with main Space Physics WebServices (CDA,SSC,AMDA,..)
Other
24 stars 7 forks source link

Option to replace cache by an exportable persistent index #122

Open jeandet opened 4 months ago

jeandet commented 4 months ago

The current cache implementation automatically updates data which is a sane default behavior. In some case, it could make sense to instead use a cache that doesn't update data and keep them forever, making a study reproducible. An API for such cache might look like this:

from speasy.reproducibility import Index
# aliasing with spz should make usage transparent 
spz = Index("/some/path")
# here the expected behavior, is, take from index if present or request data and add to index
v = spz.get_data(...)
...
# users should be able to export their cache
spz.export_data("/some/path/archive.tgz")
...
# users should be able to import cache 
spz.import_data("/some/path/archive.tgz")
...
# at some point users should also be able to update data from cache
spz.update_all_data() 
# or
v = spz.get_data(..., refresh=True)