GenomicMedLab / wags-tails

Data acquisition tools for Wagnerds
https://wags-tails.readthedocs.io/stable/
MIT License
0 stars 0 forks source link

Enable "soft version check" for date-versioned sources #35

Open jsstevenson opened 2 months ago

jsstevenson commented 2 months ago

IE, if a piece of data should be versioned as a datestamp, there should be a parameter to first check if the latest locally-available data bears today's date, and only check if there's a newer version available if it isn't.

This is sort of a "soft" median between the existing options, which are "never check against the source" and "always check against the source".

I'm interested in this for some NCBI resources, because NCBI pretty clearly throttles you if you are performing a lot of checks against their API.