fasterq-dump quit with error code 3

[que123456]

when I use prefetch, I get the following;

prefetch.3.0.1 SRR2243900

2022-12-08T13:49:10 prefetch.3.0.1: Current preference is set to retrieve SRA Normalized Format files with full base quality scores. 2022-12-08T13:49:11 prefetch.3.0.1: 1) Downloading 'SRR2243900'... 2022-12-08T13:49:11 prefetch.3.0.1: SRA Normalized Format file is being retrieved, if this is different from your preference, it may be due to current file availability. 2022-12-08T13:49:11 prefetch.3.0.1: Downloading via HTTPS... 2022-12-08T13:51:42 prefetch.3.0.1: HTTPS download succeed 2022-12-08T13:52:20 prefetch.3.0.1: 'SRR2243900' is valid 2022-12-08T13:52:20 prefetch.3.0.1: 1) 'SRR2243900' was downloaded successfully 2022-12-08T13:52:56 prefetch.3.0.1: 'SRR2243900' has 10 unresolved dependencies 2022-12-08T13:52:56 prefetch.3.0.1 int: connection unexpected while executing query within virtual file system module - cannot get remote location for 'AE013599.4'

It did download the sra file, but gives a problem at the end.

Then with fasterq-dump;

fasterq-dump.3.0.1 SRR2243900 --split-files

2022-12-08T13:53:32 fasterq-dump.3.0.1 warn: connection unexpected while executing query within virtual file system module - can't open AE014134.5 as a RefSeq or as a WGS 2022-12-08T13:53:32 fasterq-dump.3.0.1 warn: connection unexpected while executing query within virtual file system module - can't open AE014298.4 as a RefSeq or as a WGS 2022-12-08T13:53:32 fasterq-dump.3.0.1 err: cmn_iter.c cmn_read_String( #11402488 ).VCursorCellDataDirect() -> RC(rcVFS,rcQuery,rcExecuting,rcConnection,rcUnexpected) 2022-12-08T13:53:32 fasterq-dump.3.0.1 err: sorter.c get_from_raw_read_iter( 5161017 ) -> RC(rcVFS,rcQuery,rcExecuting,rcConnection,rcUnexpected) 2022-12-08T13:53:32 fasterq-dump.3.0.1 warn: connection unexpected while executing query within virtual file system module - can't open AE014296.4 as a RefSeq or as a WGS 2022-12-08T13:53:32 fasterq-dump.3.0.1 err: cmn_iter.c cmn_read_String( #22804975 ).VCursorCellDataDirect() -> RC(rcVFS,rcQuery,rcExecuting,rcConnection,rcUnexpected) 2022-12-08T13:53:33 fasterq-dump.3.0.1 warn: connection unexpected while executing query within virtual file system module - can't open AE013599.4 as a RefSeq or as a WGS 2022-12-08T13:53:33 fasterq-dump.3.0.1 err: cmn_iter.c cmn_read_String( #34207462 ).VCursorCellDataDirect() -> RC(rcVFS,rcQuery,rcExecuting,rcConnection,rcUnexpected) 2022-12-08T13:53:34 fasterq-dump.3.0.1 warn: connection unexpected while executing query within virtual file system module - can't open AE014297.2 as a RefSeq or as a WGS 2022-12-08T13:53:34 fasterq-dump.3.0.1 err: cmn_iter.c cmn_read_String( #45609949 ).VCursorCellDataDirect() -> RC(rcVFS,rcQuery,rcExecuting,rcConnection,rcUnexpected) 2022-12-08T13:53:34 fasterq-dump.3.0.1 err: sorter.c run_producer_pool().join_and_release_threads -> RC(rcExe,rcProcess,rcExecuting,rcProcess,rcCanceled) 2022-12-08T13:53:34 fasterq-dump.3.0.1 err: sorter.c execute_lookup_production() -> RC(rcExe,rcProcess,rcExecuting,rcProcess,rcCanceled) 2022-12-08T13:53:34 fasterq-dump.3.0.1 err: fasterq-dump.c produce_lookup_files() -> RC(rcExe,rcProcess,rcExecuting,rcProcess,rcCanceled) fasterq-dump quit with error code 3

and it doesn't work. so something is wrong with the RefSeqs, already when downloading with prefetch. Then it says that the connection is unexpected and it ends with error code 3.

What is wrong?

[klymenko]

What is the output of srapath AE014134.5 ?

[que123456]

2022-12-08T16:14:23 srapath.2.8.0 sys: connection failed while opening file within cryptographic module - mbedtls_ssl_handshake returned -9984 ( X509 - Certificate verification failed, e.g. CRL, CA or signature check failed ) 2022-12-08T16:14:23 srapath.2.8.0 sys: mbedtls_ssl_get_verify_result returned 0x8 ( !! The certificate is not correctly signed by the trusted CA )

[klymenko]

What is the output of the following? curl -v 'https://locate.ncbi.nlm.nih.gov/sdl/2/retrieve?acc=AE014134.5&accept-proto=https&capability=allow-cloud-refseq,ZQA:ZR'

[que123456]

that gives the following;

• Trying 165.112.7.16...
• TCP_NODELAY set
• Connected to locate.ncbi.nlm.nih.gov (165.112.7.16) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• successfully set certificate verify locations:
• CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS handshake, Certificate (11):
• TLSv1.2 (IN), TLS handshake, Server key exchange (12):
• TLSv1.2 (IN), TLS handshake, Server finished (14):
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
• TLSv1.2 (OUT), TLS change cipher, Client hello (1):
• TLSv1.2 (OUT), TLS handshake, Finished (20):
• TLSv1.2 (IN), TLS handshake, Finished (20):
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN, server accepted to use h2
• Server certificate:
• subject: CN=*.ncbi.nlm.nih.gov
• start date: Aug 2 21:41:30 2022 GMT
• expire date: Aug 2 21:41:30 2023 GMT
• subjectAltName: host "locate.ncbi.nlm.nih.gov" matched cert's "*.ncbi.nlm.nih.gov"
• issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x55e8686b0620)

  GET /sdl/2/retrieve?acc=AE014134.5&accept-proto=https&capability=allow-cloud-refseq,ZQA:ZR HTTP/2 Host: locate.ncbi.nlm.nih.gov User-Agent: curl/7.58.0 Accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 < strict-transport-security: max-age=31536000; includeSubDomains; preload < referrer-policy: origin-when-cross-origin < content-security-policy: upgrade-insecure-requests < date: Thu, 08 Dec 2022 18:00:35 GMT < server: Apache < ncbi-sid: 92BE6C1239226431_0044SID < ncbi-phid: 42540E9BD94735D500004B26CFF5A1EE.1.m_2 < content-type: application/json < cache-control: private < x-api-version: 2 < x-frame-options: DENY < set-cookie: ncbi_sid=92BE6C1239226431_0044SID; domain=.nih.gov; path=/; expires=Fri, 08 Dec 2023 18:00:35 GMT < vary: Accept-Encoding < x-ua-compatible: IE=Edge < x-xss-protection: 1; mode=block < { "version": "2", "result": [ { "bundle": "AE014134.5", "status": 200, "msg": "ok", "files": [ { "object": "refseq|AE014134.5", "accession": "AE014134.5", "type": "sra", "name": "AE014134.5", "size": 5790382, "md5": "88b96501b8d76a7c592ee6a0e3429912", "modificationDate": "2018-02-22T17:16:24Z", "locations": [ { "service": "sra-ncbi", "region": "public", "link": "https://sra-download.ncbi.nlm.nih.gov/traces/refseq/AE014134.5" } ] } ] } ] }
• Connection #0 to host locate.ncbi.nlm.nih.gov left intact

[klymenko]

What is the output of: curl -vI https://sra-download.ncbi.nlm.nih.gov/traces/refseq/AE014134.5

[que123456]

• Trying 130.14.250.27...
• TCP_NODELAY set
• Connected to sra-download.ncbi.nlm.nih.gov (130.14.250.27) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• successfully set certificate verify locations:
• CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.2 (IN), TLS handshake, Certificate (11):
• TLSv1.2 (IN), TLS handshake, Server key exchange (12):
• TLSv1.2 (IN), TLS handshake, Server finished (14):
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
• TLSv1.2 (OUT), TLS change cipher, Client hello (1):
• TLSv1.2 (OUT), TLS handshake, Finished (20):
• TLSv1.2 (IN), TLS handshake, Finished (20):
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN, server accepted to use h2
• Server certificate:
• subject: CN=*.ncbi.nlm.nih.gov
• start date: Aug 2 21:41:30 2022 GMT
• expire date: Aug 2 21:41:30 2023 GMT
• subjectAltName: host "sra-download.ncbi.nlm.nih.gov" matched cert's "*.ncbi.nlm.nih.gov"
• issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
• SSL certificate verify ok.
• Using HTTP2, server supports multi-use
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x561273646620)

  HEAD /traces/refseq/AE014134.5 HTTP/2 Host: sra-download.ncbi.nlm.nih.gov User-Agent: curl/7.58.0 Accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 HTTP/2 200 < server: nginx server: nginx < date: Thu, 08 Dec 2022 19:23:34 GMT date: Thu, 08 Dec 2022 19:23:34 GMT < content-type: application/octet-stream content-type: application/octet-stream < content-length: 5790382 content-length: 5790382 < last-modified: Thu, 22 Feb 2018 17:16:26 GMT last-modified: Thu, 22 Feb 2018 17:16:26 GMT < strict-transport-security: max-age=31536000; includeSubDomains; preload strict-transport-security: max-age=31536000; includeSubDomains; preload < content-security-policy: upgrade-insecure-requests content-security-policy: upgrade-insecure-requests < referrer-policy: origin-when-cross-origin referrer-policy: origin-when-cross-origin < allow: GET, HEAD allow: GET, HEAD < accept-ranges: bytes accept-ranges: bytes

<

• Connection #0 to host sra-download.ncbi.nlm.nih.gov left intact

[klymenko]

Please try the same command prefetch SRR2243900 from the same current directory.

[que123456]

I did, but it gives the same error as the first time

[klymenko]

What is the output of ls -l SRR2243900 from the same directory?

[que123456]

ls: cannot access 'SRR2243900': No such file or directory

[klymenko]

What is the output of srapath SRR2243900 from the same current directory?

[que123456]

/home/.../ncbi/public/sra/SRR2243900.sra

[klymenko]

What is the output of align-info SRR2243900 ?

[que123456]

2022-12-08T20:02:09 align-info.2.8.0 sys: connection failed while opening file within cryptographic module - mbedtls_ssl_handshake returned -9984 ( X509 - Certificate verification failed, e.g. CRL, CA or signature check failed ) 2022-12-08T20:02:09 align-info.2.8.0 sys: mbedtls_ssl_get_verify_result returned 0x8 ( !! The certificate is not correctly signed by the trusted CA ) 2022-12-08T20:02:09 align-info.2.8.0 int: connection failed while opening file within cryptographic module - while calling VDatabaseListDependencies: SRR2243900

[klymenko]

What is the output of the following? cd /home/.../ncbi/public/refseq/ && ls -l AE013599.4 AE014134.5 AE014135.3 AE014296.4 AE014297.2 AE014298.4 CM000457.1 CM000458.1 CM000459.1 FA000001.1

[klymenko]

And ls -l /home/.../ncbi/public/sra/SRR2243900.sra ?

[que123456]

first one gives this: -bash: cd: /home/.../ncbi/public/refseq/: No such file or directory but I could also see there is no refseq file/directory

[klymenko]

Hold on, remove toolkit version 2.8.0 from your PATH.

[que123456]

second one gives; -rw-r--r-- 1 myname domain users 3610242629 Dec 8 20:28 /home/.../ncbi/public/sra/SRR2243900.sra

[klymenko]

Did prefetch and fasterq-dump work successfully for you? When did they succeed the last time?

[que123456]

no they never worked successfully

[klymenko]

Is this the first time at all when you use sra toolkit?

[que123456]

yes

[klymenko]

Did you remove toolkit version 2.8.0 from your PATH?

[que123456]

i think so

[klymenko]

What is the output of:

align-info -V
fasterq-dump -V
prefetch -V
srapath -V

[que123456]

they all give version 3.0.1

[klymenko]

What is the output of vdb-config -on repository/user ?

[que123456]

repository/user/ad/public/apps/file/volumes/flatAd = "." repository/user/ad/public/apps/refseq/volumes/refseqAd = "." repository/user/ad/public/apps/sra/volumes/sraAd = "." repository/user/ad/public/apps/sraPileup/volumes/ad = "." repository/user/ad/public/apps/sraRealign/volumes/ad = "." repository/user/ad/public/apps/wgs/volumes/wgsAd = "." repository/user/ad/public/root = "." repository/user/default-path = "/home/myname/ncbi" repository/user/main/public/apps/file/volumes/flat = "files" repository/user/main/public/apps/nakmer/volumes/nakmerFlat = "nannot" repository/user/main/public/apps/nannot/volumes/nannotFlat = "nannot" repository/user/main/public/apps/refseq/volumes/refseq = "refseq" repository/user/main/public/apps/sra/volumes/sraFlat = "sra" repository/user/main/public/apps/wgs/volumes/wgsFlat = "wgs" repository/user/main/public/root = "/home/myname/ncbi/public"

[klymenko]

What is the output of vdb-config -on tools/prefetch/download_to_cache ?

[que123456]

download_to_cache = "true"

[que123456]

align-info SRR2243900 gives;

AE013599.4,2R,false,remote AE014134.5,2L,false,remote AE014135.3,4,false,remote AE014296.4,3L,false,remote AE014297.2,3R,false,remote AE014298.4,X,false,remote CM000456.1,2LHet,false,local CM000457.1,2RHet,false,remote CM000458.1,3LHet,false,remote CM000459.1,3RHet,false,remote CM000460.1,XHet,false,local CM000461.1,YHet,false,local FA000001.1,U,false,remote NC_001709.1,dmel_mitochondrion_genome,false,local Uextra,Uextra,false,local

[klymenko]

Try prefetch again. Does it fail?

[que123456]

It is running extremely slow, but I'll let you know when it finishes

[klymenko]

You can run prefetch with -p to see progress.

[que123456]

it still gives the same

2022-12-08T22:13:31 prefetch.3.0.1: 1) Downloading 'SRR2243900'... 2022-12-08T22:13:31 prefetch.3.0.1: SRA Normalized Format file is being retrieved, if this is different from your preference, it may be due to current file availability. 2022-12-08T22:13:31 prefetch.3.0.1: Downloading via HTTPS... |-------------------------------------------------- 100% 2022-12-08T22:19:13 prefetch.3.0.1: HTTPS download succeed 2022-12-08T22:19:44 prefetch.3.0.1: 'SRR2243900' is valid 2022-12-08T22:19:44 prefetch.3.0.1: 1) 'SRR2243900' was downloaded successfully 2022-12-08T22:20:21 prefetch.3.0.1: 'SRR2243900' has 10 unresolved dependencies 2022-12-08T22:20:21 prefetch.3.0.1 int: connection unexpected while executing query within virtual file system module - cannot get remote location for 'AE013599.4'

[que123456]

also the fasterq-dump command still gives the same output

[klymenko]

What is the output of srapath SRR2243900 from the same current directory? And srapath AE013599.4 ?

[que123456]

srapath SRR2243900; /home/myname/ncbi/public/sra/SRR2243900.sra

srapath AE013599.4; https://sra-download.ncbi.nlm.nih.gov/traces/refseq/AE013599.4

[biociao]

Came across the same issue and finally figure out that PATH setting could be the reason.

I don't know why but using $HOME instead of ~ solved my case.

To reproduce this error:

$export PATH="~/pkg/sratoolkit.3.0.2-centos_linux64/bin":$PATH
# make sure this is the only path to find sratoolkit

$fasterq-dump SRR390728
2022-12-18T09:45:50 fasterq-dump.3.0.2 err: fasterq-dump.c populate_tool_ctx().VDBManagerMakeRead() -> RC(rcFS,rcDylib,rcSearching,rcPath,rcNotFound)
fasterq-dump quit with error code **3**

$fastq-dump --stdout -X 2 SRR390728
2022-12-18T09:46:03 fastq-dump.3.0.2 err: path not found while searching dynamic library within file system module - failed to open SRA manager

=============================================================
An error occurred during processing.
A report was generated into the file '/home/myname/ncbi_error_report.txt'.
If the problem persists, you may consider sending the file
to 'sra-tools@ncbi.nlm.nih.gov' for assistance.
=============================================================

fastq-dump quit with error code 3

To solve it:

$export PATH="$HOME/pkg/sratoolkit.3.0.2-centos_linux64/bin":$PATH

$fastq-dump --stdout -X 2 SRR390728
Read 2 spots for SRR390728
Written 2 spots for SRR390728
@SRR390728.1 1 length=72
CATTCTTCACGTAGTTCTCGAGCCTTGGTTTTCAGCGATGGAGAATGACTTTGACAAGCTGAGAGAAGNTNC
+SRR390728.1 1 length=72
;;;;;;;;;;;;;;;;;;;;;;;;;;;9;;665142;;;;;;;;;;;;;;;;;;;;;;;;;;;;;96&&&&(
@SRR390728.2 2 length=72
AAGTAGGTCTCGTCTGTGTTTTCTACGAGCTTGTGTTCCAGCTGACCCACTCCCTGGGTGGGGGGACTGGGT
+SRR390728.2 2 length=72
;;;;;;;;;;;;;;;;;4;;;;3;393.1+4&&5&&;;;;;;;;;;;;;;;;;;;;;<9;<;;;;;464262

Use absolute path also works.

[klymenko]

@que123456, do you still need help?