Incremental loading - Githubissues

EBISPOT / eqtl-sumstats-service

eQTL Summary Statistics Service

0 stars 0 forks source link

Incremental loading #14

Closed karatugo closed 1 month ago

karatugo commented 2 months ago

[x] Create a new Mongo document where all the status are kept
- [x] Possible keys: Study, Dataset, Status, Date
[x] Remove sync.log and the logic around that

karatugo commented 2 months ago

PR merged, see above. Now we keep all study-dataset-filename-status in a Mongo collection. Therefore incremental loading is possible whenever new files are added.