Create ingest cache - Githubissues

monarch-initiative / monarch-ingest

Data ingest application for Monarch Initiative knowledge graph using Koza

https://monarchinitiative.org

14 stars 1 forks source link

Closed kevinschaper closed 1 year ago

kevinschaper commented 2 years ago

This should download from original sources, updating the cache only if the file is newer and passes validation checks.

Potential validation checks:

File is not empty
Checksum
Koza validate against monarch ingest config (do we cache a file even if we need to update the ingest? maybe yes?)

kevinschaper commented 2 years ago

I'm going to icebox the cache. It's very likely that we'll still want a DipperCache equivalent, but I don't think it's absolutely essential.

kevinschaper commented 1 year ago

Oh my, we've had the cache for some time. This is done! 🥳