s-andrews
commented
4 years ago An example:
Downloading SRR8611939 into SRR8611939_GSM3618718_HUES8_WT_WGBS_Homo_sapiens_Bisulfite-Seq
Running: fasterq-dump --split-files --threads 1 --outfile SRR8611939_GSM3618718_HUES8_WT_WGBS_Homo_sapiens_Bisulfite-Seq
--progress SRR8611939
lookup :| 0.81%2020-06-12T13:58:27 fasterq-dump.2.10.7 err: cmn_iter.c cmn_read_String( #427700225 ).VCursorCellDataDirect() -> RC(rcPS,rcCondition,rcWaiting,rcTimeout,rcExhausted)
2020-06-12T13:58:27 fasterq-dump.2.10.7 err: sorter.c write_to_store().pack_read_2_4na() failed RC(rcVDB,rcNoTarg,rcWriting,rcFormat,rcNull)
2020-06-12T13:58:27 fasterq-dump.2.10.7 err: cmn_iter.c cmn_read_String( #4244481 ).VCursorCellDataDirect() -> RC(rcPS,rcCondition,rcWaiting,rcTimeout,rcExhausted)
2020-06-12T13:58:27 fasterq-dump.2.10.7 err: sorter.c write_to_store().pack_read_2_4na() failed RC(rcVDB,rcNoTarg,rcWriting,rcFormat,rcNull)
2020-06-12T14:18:12 fasterq-dump.2.10.7 err: transfer canceled while allocating buffer within file system module - Cannot KHttpFileTimedReadChunked: to=480
2020-06-12T14:18:12 fasterq-dump.2.10.7 err: sorter.c run_producer_pool().join_and_release_threads -> RC(rcVDB,rcNoTarg,rcWriting,rcFormat,rcNull)
2020-06-12T14:18:12 fasterq-dump.2.10.7 err: sorter.c execute_lookup_production() -> RC(rcVDB,rcNoTarg,rcWriting,rcFormat,rcNull)
merge :
2020-06-12T14:18:12 fasterq-dump.2.10.7 err: fasterq-dump.c produce_lookup_files() -> RC(rcVDB,rcNoTarg,rcWriting,rcFormat,rcNull)
fasterq-dump quit with error code 3
Traceback (most recent call last):
File "/bi/home/andrewss/sradownloader/sradownloader", line 199, in <module>
main()
File "/bi/home/andrewss/sradownloader/sradownloader", line 196, in main
download_sample(sample,options)
File "/bi/home/andrewss/sradownloader/sradownloader", line 70, in download_sample
subprocess.run(command_options, check=True)
File "/bi/apps/python/3.7.3/lib/python3.7/subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['fasterq-dump', '--split-files', '--threads', '1', '--outfile', 'SRR8611939_GSM3618718_HUES8_WT_WGBS_Homo_sapiens_Bisulfite-Seq\n', '--progress', 'SRR8611939']' returned non-zero exit status 3.
Despite their protestations fasterq-dump has a pretty awful failure rate and just crashes and burns in a non-reproducible way.
Can we set up a retry system so that we don't lose a whole download set if it's having a bad day.