search

rvalieris / parallel-fastq-dump

parallel fastq-dump wrapper
MIT License
275 stars 33 forks source link

Something writes to root, even if tmpdir is defined. #26

Closed joachimwolff closed 4 years ago

joachimwolff commented 4 years ago

Hi,

This is a very nice tool to speed-up the download via fastq-dump.

I have the issue that fastq-dump has sometimes errors like: timeout exhausted while waiting condition within process system module

a) Do you know what this error is? It seems to have some relation to the internet speed: on my local computer with ~50Mbits it raises; however if I download it via my local compute center and a 1 Gbits connection this error does not occure.

b) Using the fast connection (1 Gbit) I am facing another problem. I defined a tmp dir in my home but still something is written to my root. And then I get 2020-03-08T11:06:02 fastq-dump.2.10.3 err: storage exhausted while writing file within file system module - system bad file descriptor error fd='4'.

parallel-fastq-dump --sra-id SRR5229019 -t 40 --outdir out_fastq/ --tmpdir local_tmp

2020-03-08_12-19

Any idea how to solve the two issues?

For the first one: If it is an issue of fastq-dump, would it be possible to have sth which is restarting the download from the point it crashed?

Best,

Joachim

rvalieris commented 4 years ago

hello,

a) I don't know about this specific message, but it should be fine as long as the download finish ok, always check if the final file have the expected number of reads.

b) recent versions of sra-tools changed the default cache location to /var/tmp, to change this location use vdb-config -i navigate to cache and set both locations to a path that have enough space.