AstrobioMike
commented
1 year ago Hi there, @corkdagga!
Thanks for the note. There is indeed no current way to set that for the test run, which was a silly oversight of me to make. I will add that in ASAP. But updating to a newer version might help. Do you know if there's a reason it was using v1.6.34? We're currently up to v1.8.1, and there have been some improvements to the downloading and download-error handling since then that I think might solve this.
If you wouldn't mind, i'd be interested to know if it happens still if trying with the latest version. You should be able to install it with the following (modified environment name just to not overwrite your current one):
# if you don't have mamba already
conda install -n base -c conda-forge mamba
# creating new gtotree env with v1.8.1
mamba create -n gtotree-1.8.1 -c astrobiomike -c conda-forge -c bioconda -c defaults gtotree=1.8.1
conda activate gtotree-1.8.1
gtt-test.shI'll update you when i add the ability to set HTTP with the test run, thanks for the note about that!
corkdagga
Hi,
I have just iinstalled Gtotree on a HPC using conda.
After running gtt-test.sh, I get the following output (below). I guess it is some kind of connection issue because there are multiple errors. I noticed issue #32 when trying to download from NCBI, the second option in issue #32 worked for me, but I have no idea how to apply that to gtt-test.sh, since the commands are run automatically. Is there some kind of option or commands I can input to permanently fix my issue?
Any help here would be great! Thanks!
(gtotree) pada358b@tauruslogin5:/beegfs/.global0/ws/pada358b-conda$ gtt-test.sh
Downloading GToTree test data into the subdirectory GToTree-test-data/
Data being pulled from here: https://figshare.com/articles/dataset/GToTree_test_data/19372334
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7163k 100 7163k 0 0 9339k 0 --:--:-- --:--:-- --:--:-- 24.7M
Running test as: GToTree -a GToTree-test-data/ncbi_accessions.txt \ -g GToTree-test-data/genbank_files.txt \ -f GToTree-test-data/fasta_files.txt \ -A GToTree-test-data/amino_acid_files.txt \ -m GToTree-test-data/genome_to_id_map.tsv \ -p GToTree-test-data/pfam_targets.txt \ -H Universal -t -D -j 4 -o GToTree-test-output
The test run includes some things that shouldn't be found, so don't be alarmed when seeing those messages.
Starting run now:
Traceback (most recent call last): File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 1566, in ftp_open fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout) File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 1588, in connect_ftp persistent=False) File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 2408, in init self.init() File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 2417, in init self.ftp.connect(self.host, self.port, self.timeout) File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/ftplib.py", line 154, in connect source_address=self.source_address) File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/socket.py", line 728, in create_connection raise err File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/socket.py", line 716, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/beegfs/ws/0/pada358b-conda/envs/gtotree/bin/gtt-get-ncbi-tax-data", line 123, in
main()
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/bin/gtt-get-ncbi-tax-data", line 41, in main
get_NCBI_tax_data(NCBI_data_dir)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/bin/gtt-get-ncbi-tax-data", line 110, in get_NCBI_tax_data
urllib.request.urlretrieve("ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz", taxdump_path)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 1584, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 1566, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 1588, in connect_ftp
persistent=False)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 2408, in init
self.init()
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/urllib/request.py", line 2417, in init
self.ftp.connect(self.host, self.port, self.timeout)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/ftplib.py", line 154, in connect
source_address=self.source_address)
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/socket.py", line 728, in create_connection
raise err
File "/beegfs/ws/0/pada358b-conda/envs/gtotree/lib/python3.7/socket.py", line 716, in create_connection
sock.connect(sa)
urllib.error.URLError: <urlopen error ftp error: ConnectionRefusedError(111, 'Connection refused')>
--------------------------------- RUN INFO ---------------------------------
** NOTICE **
Filtering by gene-length using the median length of a gene set (set with the
-cflag) becomes less reliable with fewer genomes. With 14 total input genomes, if a lot of sequences are dropped, consider increasing the parameter and/or visually inspecting the alignments.##############################################################################
Downloading HMMs for additional Pfam targets
##############################################################################
/beegfs/ws/0/pada358b-conda/envs/gtotree/bin/gtt-get-additional-pfam-targets.sh: line 22: [: PF00238.19: unary operator expected cat: 1690194970.gtotree.tmpdir/.hmm: No such file or directory /beegfs/ws/0/pada358b-conda/envs/gtotree/bin/gtt-get-additional-pfam-targets.sh: line 22: [: PF05400.13: unary operator expected cat: 1690194970.gtotree.tmpdir/.hmm: No such file or directory
##############################################################################
Working on the genomes provided as NCBI accessions
##############################################################################
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (7) Failed to connect to ftp.ncbi.nlm.nih.gov port 21 after 112 ms: Connection refused
Download of NCBI assembly summaries failed :( Is the internet connection weak?
Exiting for now.
Test completed! See here for how things should look: https://github.com/AstrobioMike/GToTree/wiki/Installation#test-run
You can clear out the test data and results by running: gtt-clean-after-test.sh
(gtotree) pada358b@tauruslogin5:/beegfs/.global0/ws/pada358b-conda$