metaspace2020 / Lithops-METASPACE

Lithops-based Serverless implementation of the METASPACE spatial metabolomics annotation pipeline
12 stars 4 forks source link

Improve dataset sorting script #70

Closed omerb01 closed 4 years ago

omerb01 commented 4 years ago

Improve the way that the script downloads the dataset. with this change, the download will be in chunks of 1GB that will be loaded into memory and then written into disc.