metno / aerovaldb

aerovaldb is the database-interface used in the aeroval-API to send data to aeroval database instances like https://aeroval.met.no . Data is usually provided by PyAerocom.
https://aerovaldb.readthedocs.io
GNU General Public License v3.0
1 stars 1 forks source link

Cache uri generation in json #89

Open thorbjoernl opened 2 months ago

thorbjoernl commented 2 months ago

Change Summary

Adds persistent caching of URIs to the jsonfiledb implementation, which speeds up iterating over uris significantly (once they have been cached).

INFO:aerovaldb.jsondb.jsonfiledb:Initializing aerovaldb for './tests/test-db/json' with locking False
json_files: 47 items in 0.117 seconds
sqlite:     47 items in 0.001 seconds
INFO:aerovaldb.jsondb.jsonfiledb:Initializing aerovaldb for './tests/test-db/json' with locking False
json_files: 47 items in 0.004 seconds (Cached)

Related issue number

74

Checklist