thespacedoctor / sherlock

The QUB Transient Classifier
GNU General Public License v3.0
11 stars 5 forks source link

Build a NED caching code that runs continually instead of on-the-fly caching #145

Open thespacedoctor opened 2 years ago

thespacedoctor commented 2 years ago

Currently Sherlock can pull down a local cache of nearby NED sources for each transient it is presented with, but this caching stage is often skipped as it adds a very considerable overhead to the Sherlock run time. This means that Sherlock needs to rely on the pre-existing local cache which can be sparse in many sky-areas (see below).

I need to come up with a methodical technique to cache the entire NED database locally by allowing the data to trickle in slowly over time.