search

rvalieris / parallel-fastq-dump

parallel fastq-dump wrapper
MIT License
265 stars 33 forks source link

Failed to retrieve results error #24

Closed JonathonMifsud closed 4 years ago

JonathonMifsud commented 4 years ago

Hi,

I am running into this error - I installed parallel-fastq-dump on an AWS EC2 using conda. Any insights would be wonderful. I suspect this may be an error with fastq dump itself but I could not find any information on it either.

My code: parallel-fastq-dump --sra-id ERS1829683 --threads 56 --outdir ./ --split-files Output: SRR ids: ['ERS1829683'] extra args: ['--split-files'] tempdir: /tmp/pfd_9bpf3fvs ERS1829683 spots: 13339116 blocks: [[1, 238198], [238199, 476396], [476397, 714594], [714595, 952792], [952793, 1190990], [1190991, 1429188], [1429189, 1667386], [1667387, 1905584], [1905585, 2143782], [2143783, 23819 80], [2381981, 2620178], [2620179, 2858376], [2858377, 3096574], [3096575, 3334772], [3334773, 3572970], [3572971, 3811168], [3811169, 4049366], [4049367, 4287564], [4287565, 4525762], [4525 763, 4763960], [4763961, 5002158], [5002159, 5240356], [5240357, 5478554], [5478555, 5716752], [5716753, 5954950], [5954951, 6193148], [6193149, 6431346], [6431347, 6669544], [6669545, 69077 42], [6907743, 7145940], [7145941, 7384138], [7384139, 7622336], [7622337, 7860534], [7860535, 8098732], [8098733, 8336930], [8336931, 8575128], [8575129, 8813326], [8813327, 9051524], [9051 525, 9289722], [9289723, 9527920], [9527921, 9766118], [9766119, 10004316], [10004317, 10242514], [10242515, 10480712], [10480713, 10718910], [10718911, 10957108], [10957109, 11195306], [111 95307, 11433504], [11433505, 11671702], [11671703, 11909900], [11909901, 12148098], [12148099, 12386296], [12386297, 12624494], [12624495, 12862692], [12862693, 13100890], [13100891, 1333911 6]] failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result fastq-dump error! exit code: 22 failed to retrieve result (virus) jmifsud@dacelo:/disks/dacelo/data/jmifsud$ failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result failed to retrieve result Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621 Read 238198 spots for ERR2040621 Written 238198 spots for ERR2040621

rvalieris commented 4 years ago

hello, I can't replicate the error here, looks like an issue with fastq-dump, have you tried running just the plain fastq-dump ?

Written 238198 spots for ERR2040621

also looks like the files are getting written, just a guess but maybe 56 threads is too much ?

JonathonMifsud commented 4 years ago

Hi,

Thank you for your response. Plain fastq-dump works fine - 56 threads is within the servers capacity but I have tried a lower number and still run into this error.

Two more errors appear when I let it run further:

fastq-dump.2.10.0 err: path not found while creating directory within file system module - failed ERR2040409 fastq-dump error! exit code: 22 Interestingly ERR2040409 wasn't even the accession I was using. I just noticed with the example I included above the Written accession (ERR2040409) is different to the one I entered (ERS1829683)

EDIT: I have a similar set up on another server where parallel works fine. When I check the parallel-fastq version I noticed that under parallel-fastq 0.6.5, fastq-dump : 2.9.6 is present on the working server but not on this server. Both have sra-tools installed, the working one is 2.9.6 while the broken one is 2.10.0. Apologizes if this is more to do with fastq-dump.

JonathonMifsud commented 4 years ago

Reverting sra-tools from 2.10.0 to 2.9.6 worked for me. Thanks again!

rvalieris commented 4 years ago

nice, thanks for the update.

Interestingly ERR2040409 wasn't even the accession I was using. I just noticed with the example I included above the Written accession (ERR2040409) is different to the one I entered (ERS1829683)

ERS/SRS is the ID of a sample, ERR/SRR is the ID of a run, which is what will be downloaded. you can check see a list of all runs of a sample like this: https://www.ncbi.nlm.nih.gov/sra/?term=ERS1829683