search

IGS / portal_client

Python-based client for downloading data made available through portals powered by the GDC-based portal system..
MIT License
16 stars 17 forks source link

Error using manifest file, error message: Skipping file ID <file_id> as none of the URLs ['FASP'] succeeded. #12

Closed ishworthapa closed 4 years ago

ishworthapa commented 4 years ago

Hi, I am executing the portal_client tool (with my iHMP username and password) to download the files using a manifest file as follows: portal_client -m hmp_cart_171723e34.tsv --endpoint-priority FASP --user myusername

However, I get an error message as follows:

Skipping file ID 216b55a2f235085126ee6210d619d9fa as none of the URLs ['FASP'] succeeded.
Skipping file ID 1419f08f554e0c93f3b62fe90c092ab8 as none of the URLs ['FASP'] succeeded.
Skipping file ID 1419f08f554e0c93f3b62fe90c092760 as none of the URLs ['FASP'] succeeded.

Not all files (total of 3) were downloaded successfully. Number of failures:
0 -- no valid URL in the manifest file
3 -- URL is present in manifest, but not accessible at the location specified
0 -- MD5 checksum failed for file (file is corrupted or the wrong MD5 is associated)

The content of the manifest file is below: file_id md5 size urls sample_id 216b55a2f235085126ee6210d619d9fa 1e82180a7680651167619cf407e889d3 686599 fasp://aspera2.ihmpdcc.org/ibd/transcriptome/host/analysis/CSM5FZ1I.tsv 7cfd74d6803ea319683df75644d2ba27 1419f08f554e0c93f3b62fe90c092ab8 23eb8207442bd452f0d77ecc0fdba7f9 24137 fasp://aspera2.ihmpdcc.org/ibd/genome/microbiome/wgs/analysis/hmscp/CSM5MCXD_taxonomic_profile.biom 1419f08f554e0c93f3b62fe90c08f0f2 1419f08f554e0c93f3b62fe90c092760 c761814a69f7a5cec26859c945e6ccde 141691 fasp://aspera2.ihmpdcc.org/ibd/genome/microbiome/wgs/analysis/hmmrc/CSM5MCXD_pathabundance_relab.biom 1419f08f554e0c93f3b62fe90c08f0f2

Can you please check if my command is correct? I also tried using ascp directly but it didn't give me any output either.

Thanks, Ishwor

victor73 commented 4 years ago

In your manifest (the tsv file), please try replacing "aspera2" in the URLs with just "aspera", and try again.

ishworthapa commented 4 years ago

It worked! Thank you for your help.

Thanks, Ishwor

nano1231 commented 3 years ago

Hi I get the same error with the same manifest format but my URLs are as follows:

file_id | md5 | size | urls | sample_id

e85a36182291e14f07e89efdce0e3b91 | 743aefa4476140f13ae8657b56f528b3 | 2582578 | https://downloads.hmpdacc.org/ihmp/ibd/genome/microbiome/16s/raw/206547.fastq.gz | 216b55a2f235085126ee6210d60a61be

Thanks in advance. Fernando

hsbi commented 3 years ago

Same as Fernando, "aspera2" is not in any URLs in the manifest .tsv file I have.

All the failures are of the category: "URL is present in manifest, but not accessible at the location specified"

Sample of Manifest URLs: file_id md5 size urls sample_id 1419f08f554e0c93f3b62fe90c570fb6 84019f57facdaab2df6900205b99e21e 1067898880 https://downloads.hmpdacc.org/ihmp/ibd/genome/microbiome/wgs/raw/CSM79HNU.tar 1419f08f554e0c93f3b62fe90c56f0fe

nano1231 commented 3 years ago

Hi Yujin I tried that and still didn't work. Is not the urls because I can download individual files. If you find out anything else please share. thank you, Fernando

Fernando E. Nieto-Fernandez, PhD Professor of Biology and STEP Program Director SUNY Old Westbury Partnership for Undergraduate Life Sciences Education (PULSE) Fellow https://www.oldwestbury.edu/people/nietof https://sites.google.com/view/sunyowstep/home https://www.facebook.com/STEP-Program-at-SUNY-Old-Westbury-251520886716/ @STEPOldWestbury

Pronouns: he, him, his Pronombres: el, su, suyo.

The land on which I live and work is the unceded territory of the Lenape, including the Canarsee and the Nayack, and Matinecock. I pay respect to Indigenous communities and elders, past, present and future, and will work towards dismantling the legacies of settler colonialism.

From: Yujin Chung @.> Sent: Thursday, July 8, 2021 6:19 AM To: IGS/portal_client @.> Cc: Fernando Nieto @.>; Comment @.> Subject: Re: [IGS/portal_client] Error using manifest file, error message: Skipping file ID as none of the URLs ['FASP'] succeeded. (#12)

It worked for me after removing the last column, 'sample_id'.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FIGS%2Fportal_client%2Fissues%2F12%23issuecomment-876318409&data=04%7C01%7Cnietof%40oldwestbury.edu%7C26fc792a8c4245df024e08d941f9de97%7Cf5089034f2334f12a71638b7f4904370%7C0%7C0%7C637613363713924162%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CvvUOgH0jR1vGr20vx3qowHMDr4Bz75lQj5PgJLBrMg%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPRMHFYW3L4QUGTIC4F64QTTWV3S5ANCNFSM4I42MIHA&data=04%7C01%7Cnietof%40oldwestbury.edu%7C26fc792a8c4245df024e08d941f9de97%7Cf5089034f2334f12a71638b7f4904370%7C0%7C0%7C637613363713934117%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=F1peQXAMnu3XpMV5dCsl5s0AhyNkJ6JKQ06YnhROlEQ%3D&reserved=0.