Open schorlton opened 4 years ago
I also see this issue when downloading a large amount of data, and the script doesn't let you resume, so it's downloading right from the start again.
I have added a --timeout
parameter. Increasing this will hopefully fix the observed issue.
Medium-term, it would be good to add a --resume
parameter as well, or an automatic check whether an assembly has been downloaded already. Doing this should not be too complicated (checking whether the files in @genomic_fna_files
and @assembly_report_files
are present already).
Hi Alexander,
Good timing as I've just been playing with some code to resume. I'll make a pull request when I've got something tested.
Cheers!
Rob
From: Alexander Dilthey notifications@github.com Sent: Thursday, November 14, 2019 3:09:05 PM To: DiltheyLab/MetaMaps MetaMaps@noreply.github.com Cc: robert davey (EI) robert.davey@earlham.ac.uk; Comment comment@noreply.github.com Subject: Re: [DiltheyLab/MetaMaps] downloadRefseq.pl timeout (#18)
I have added a --timeout parameter. Increasing this will hopefully fix the observed issue.
Medium-term, it would be good to add a --resume parameter as well, or an automatic check whether an assembly has been downloaded already. Doing this should not be too complicated (checking whether the files in @genomic_fna_files and @assembly_report_files are present already).
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DiltheyLab/MetaMaps/issues/18?email_source=notifications&email_token=AAOJDLXQ3NFTNFVBM54RGVLQTVSZDA5CNFSM4I5V7KQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEECEUYQ#issuecomment-553929314, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOJDLXQND74YNE7PPCSVNLQTVSZDANCNFSM4I5V7KQA.
Hi Rob,
Wonderful! Thank you! Looking forward to the pull request! :-)
Cheers,
Alex
Hi,
I am having timeout issue. Tried increasing the --timeout several times but it still doesn't work. @AlexanderDilthey, would appreciate if you could suggest how to get this resolved.
perl $METAMAPS_SRC_DIR/downloadRefSeq.pl --targetBranches archaea,bacteria,fungi,plant,protozoa --skipIncompleteGenomes 1 --seqencesOutDirectory $METAMAPS_DB_PATH/download/refseq --taxonomyOutDirectory $METAMAPS_DB_PATH/download/taxonomy --timeout 720
Few errors encountered: 1) Timeout at /util/common/bioconda/20200706/anaconda-py37/lib/5.26.2/Net/FTP.pm line 583 2) Net::FTP=GLOB(0x5598d583cf60)<<< 421 Idle timeout (60 seconds): closing control connection
Thank you. -Tanushri
Running from the latest commit (
881373c
). Tried downloading the database. Unfortunately timing out as per below.