search

InseeFrLab / pynsee

pynsee package contains tools to easily search and download french data from INSEE and IGN APIs
https://pynsee.readthedocs.io/en/latest/
MIT License
70 stars 10 forks source link

Should macrodata.get_series use cache? #195

Closed tfardet closed 5 months ago

tfardet commented 6 months ago

Is it normal that get_series does not cache results? Would it be possible to check for the periodicity of the requested data via get_series_list and only query if the last query is older than the periodicity? Unless there are really data that are updated several times a day, we could even have at least a default caching duration of one day before we query the API again (many series have annual or monthly updates and querying them everytime seems like a shame).

hadrilec commented 6 months ago

Hi, indeed the caching was originally thought to be used only on metadata. If you think that as a user you would appreciate to cache the quantitative data from get_series, then fine I will implement this feature as well.

tfardet commented 6 months ago

If it's not too much trouble, I think it would make sense to have a small file/db telling when the data was last downloaded and update after a certain period. Either you what's the fastest update rate for data and you just set it to that for everything, or you can easily get the refresh rate for each idBank and set it per idBank to match the update rate?