search

cumc / xqtl-protocol

Molecular QTL analysis protocol developed by ADSP Functional Genomics Consortium
https://cumc.github.io/xqtl-protocol/
MIT License
36 stars 42 forks source link

Problem with download_hg_reference #383

Open hsun3163 opened 1 year ago

hsun3163 commented 1 year ago

While running sos run pipeline/reference_data.ipynb download_hg_reference --cwd reference_data in the login node, following error occurs.

hs3163@csglogin:/mnt/vast/hpc/csg/xqtl_workflow_testing/finalizing/reference_data$
GRCh38_ful...lus_decoy_hla.fa: <urlopen error ftp error: TimeoutError(110, 'Connection timed out')>:
ERROR: download_hg_reference (id=c29278acf831fdde) returns an error.
ERROR: [download_hg_reference]: [0]:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
script_4737362758465142950 in <module>
----> download('ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa\n\n', dest_dir = cwd)

RuntimeError: Failed to download {urls[0]}

[1]+  Exit 1                  sos run pipeline/reference_data.ipynb download_hg_reference --cwd reference_data  (wd: /mnt/vast/hpc/csg/xqtl_workflow_testing/finalizing)
hsun3163 commented 1 year ago

However, while not running the sos wrapper, but instead downloads the link directly via wget ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa &, it works:

hs3163@csglogin:/mnt/vast/hpc/csg/xqtl_workflow_testing/finalizing/reference_data$ --2022-09-08 18:44:44--  http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa
Resolving bcp3.cumc.columbia.edu (bcp3.cumc.columbia.edu)... 10.139.211.159
Connecting to bcp3.cumc.columbia.edu (bcp3.cumc.columbia.edu)|10.139.211.159|:8080... connected.
Proxy request sent, awaiting response...
hs3163@csglogin:/mnt/vast/hpc/csg/xqtl_workflow_testing/finalizing/reference_data$
hs3163@csglogin:/mnt/vast/hpc/csg/xqtl_workflow_testing/finalizing/reference_data$ 200 OK
Length: 3263683042 (3.0G)
Saving to: 'GRCh38_full_analysis_set_plus_decoy_hla.fa'

               GRCh38_full_an   0%[                                                  ]       0  --.-KB/s
   GRCh38_full_analysis_set_p   0%[                                                  ]  11.05M   655KB/s    eta 81m 21s