search

rvalieris / parallel-fastq-dump

parallel fastq-dump wrapper
MIT License
265 stars 33 forks source link

it never ends... #39

Closed sunnlife1978 closed 3 years ago

sunnlife1978 commented 3 years ago

Hello, I started a download with the following command: parallel-fastq-dump --sra-id SRR9089604 --threads 10 --outdir out/ --tmpdir temp/ This SRR is about 80Gb. The temp folder is already more than 100Gb downloaded and it continues... is it normal?

thanks for your help

Alex

rvalieris commented 3 years ago

hello, 2 things:

  1. to download the data, its faster if you use the prefetch command, after downloading you can use fastq-dump to convert the sra to fastq as you are doing.

  2. use the --gzip flag, fastq compresses very well with gzip, without this flag you are wasting disk space.

the SRR is 80GB compressed, since you're not using --gzip, those 100G in the temp folder represent about 40-50% of completion.