Uncompress SARS2 feed file on the fly to save disk space

vector-engineering / covidcg

A COVID-19 CoV Genetics (CG) browser to inform therapeutics development

https://covidcg.org

MIT License

26 stars 5 forks source link

Closed atc3 closed 11 months ago

atc3 commented 1 year ago

Decompress the feed.json.xz file as a stream in process_feed.py instead of storing the plaintext JSON on disk -- it was exceeding hundreds of GB