search

ncbi / sra-tools

SRA Tools
Other
1.1k stars 242 forks source link

fastq-dump error! exit code: 3 #953

Closed binbinL closed 1 month ago

binbinL commented 1 month ago

Hi, I have downloaded SRR files and now tend to use parallel-fastq-dump convert them to .fastq.gz files when i run "parallel-fastq-dump -t 4 -O ./fastq --split-files --gzip -s /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793" under /tmp dir

error ： 2024-07-25 16:13:51,267 - SRR ids: ['/public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793'] 2024-07-25 16:13:51,267 - extra args: ['--split-files', '--gzip'] 2024-07-25 16:13:51,268 - tempdir: /tmp/pfd_wyozb3vj 2024-07-25 16:13:51,268 - CMD: sra-stat --meta --quick /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25 16:13:51,292 - /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 spots: 34906477 2024-07-25 16:13:51,292 - blocks: [[1, 8726619], [8726620, 17453238], [17453239, 26179857], [26179858, 34906477]] 2024-07-25 16:13:51,292 - CMD: fastq-dump -N 1 -X 8726619 -O /tmp/pfd_wyozb3vj/0 --split-files --gzip /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25 16:13:51,293 - CMD: fastq-dump -N 8726620 -X 17453238 -O /tmp/pfd_wyozb3vj/1 --split-files --gzip /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25 16:13:51,293 - CMD: fastq-dump -N 17453239 -X 26179857 -O /tmp/pfd_wyozb3vj/2 --split-files --gzip /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25 16:13:51,294 - CMD: fastq-dump -N 26179858 -X 34906477 -O /tmp/pfd_wyozb3vj/3 --split-files --gzip /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25T08:13:51 fastq-dump.2.8.0 err: column not found while opening table within short read archive module - failed /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25T08:13:51 fastq-dump.2.8.0 err: column not found while opening table within short read archive module - failed /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25T08:13:51 fastq-dump.2.8.0 err: column not found while opening table within short read archive module - failed /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 2024-07-25T08:13:51 fastq-dump.2.8.0 err: column not found while opening table within short read archive module - failed /public/home/mengqx/dengxsh/WorkSpace/Data/tmp/SRR18141793 ... fastq-dump error! exit code: 3

Version parallel-fastq-dump : 0.6.7 fastq-dump : 2.8.0

Thanks!

wraetz commented 1 month ago

Hi, parallel-fastq-dump is not developed by or supported by us. It is a 3rd party script - I do not know the origin of. If you are interested in a faster version of fastq-dump : download the newest toolkit form the NIH-website ( https://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/3.1.1/ ) and use the fasterq-dump tool. Run 'fasterq-dump -h' first - some of the options and defaults a different from fastq-dump. The new tool uses multiple threads internally ( and a temporary lookup-table where it helps ) to speed up the conversion from SRR to FASTQ/FASTA.

klymenko commented 1 month ago

First of all - update the toolkit.

binbinL commented 1 month ago

I updated version: parallel-fastq-dump : 0.6.7 fastq-dump : 3.1.0

Hi, parallel-fastq-dump is not developed by or supported by us. It is a 3rd party script - I do not know the origin of. If you are interested in a faster version of fastq-dump : download the newest toolkit form the NIH-website ( https://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/3.1.1/ ) and use the fasterq-dump tool. Run 'fasterq-dump -h' first - some of the options and defaults a different from fastq-dump. The new tool uses multiple threads internally ( and a temporary lookup-table where it helps ) to speed up the conversion from SRR to FASTQ/FASTA.

ok thanks~

binbinL commented 1 month ago

First of all - update the toolkit.

I updated version: parallel-fastq-dump : 0.6.7 fastq-dump : 3.1.0

then err： data corrupt while executing function within virtual database module - failed fastq-dump quit with error code 3

https://github.com/ncbi/sra-tools/wiki/FAQ#faq4 Some say it is the reason for the bad data?

klymenko commented 1 month ago

Run prefetch first: see https://github.com/ncbi/sra-tools/wiki/08.-prefetch-and-fasterq-dump