Bulk download failing to download large numbers of files

EBI-Metagenomics / emg-toolkit

MGnify API toolkit

https://www.ebi.ac.uk/metagenomics

Apache License 2.0

21 stars 4 forks source link

Bulk download failing to download large numbers of files #15

Closed tomblaze closed 3 years ago

tomblaze commented 3 years ago

bulk_download.py currently doesn't process all available files due to relatively simple count-iteration bug / typo. I made a pull request #14 that I think should fix the issue.

Example of run that failed: mg-toolkit -d bulk_download -p 5.0 -a MGYS00002401 -g taxonomic_analysis_ssu_rrna

Stops after downloading 150 files as num_results_processed is incremented cumulatively in each loop and 25 + 50 + 75 + 100 + 125 + 150 = 525 which is greater than the total number of files to download.

mberacochea commented 3 years ago

Hi Tom,

Thank you for reporting the bug and submitting the fix!. I'll merge this and release a new version.

Cheers

mberacochea commented 3 years ago

Merged and released. Thank you