search

nextgenusfs / funannotate

Eukaryotic Genome Annotation Pipeline
http://funannotate.readthedocs.io
BSD 2-Clause "Simplified" License
322 stars 85 forks source link

Funannotate setup dbCAN database fails to download #285

Closed mswyers closed 5 years ago

mswyers commented 5 years ago

Are you using the latest release? If you are not using the latest release of funannotate, please upgrade, if bug persists then report here.

Describe the bug Funannotate setup fails on downloading dbCAN database.

What command did you issue? funannotate setup -d /PATH/TO/DB/

Logfiles

[09:51 AM]: OS: linux2, 8 cores, ~ 33 GB RAM. Python: 2.7.16
[09:51 AM]: Running funannotate v1.5.2
[09:51 AM]: Database location: home/seq/fundb/
[09:51 AM]: Downloading Merops database
[09:51 AM]: Downloading: ftp://ftp.ebi.ac.uk/pub/databases/merops/current_release/merops_scan.lib Bytes: 1942655
[09:51 AM]: Building diamond database
[09:51 AM]: MEROPS Database: version=12.0 date=2017-10-04 records=4,968
[09:51 AM]: Downloading UniProtKB/SwissProt database
[09:51 AM]: Downloading: ftp://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz Bytes: 88495193
[09:51 AM]: Downloading: ftp://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/reldate.txt Bytes: 151
[09:51 AM]: Building diamond database
[09:52 AM]: UniProtKB Database: version=2019_03 date=2019-04-10 records=559,634
[09:52 AM]: Downloading dbCAN database

Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/Cellar/funannotate/1.5.2/libexec/bin/funannotate-setup.py", line 443, in <module>
    dbCANDB(DatabaseInfo, args.force)
  File "/home/linuxbrew/.linuxbrew/Cellar/funannotate/1.5.2/libexec/bin/funannotate-setup.py", line 162, in dbCANDB
    download(lib.DBURL.get('dbCAN'), os.path.join(FUNDB,'dbCAN.tmp'))
  File "/home/linuxbrew/.linuxbrew/Cellar/funannotate/1.5.2/libexec/bin/funannotate-setup.py", line 67, in download
    u = urllib2.urlopen(url)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 1228, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/urllib2.py", line 1198, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>
$ funannotate check --show-versions
-------------------------------------------------------
Checking dependencies for funannotate v1.5.2
-------------------------------------------------------
You are running Python v 2.7.16. Now checking python packages...
biopython: 1.73
goatools: 0.8.12
matplotlib: 2.2.3
natsort: 5.5.0
numpy: 1.15.4
pandas: 0.23.4
psutil: 5.4.8
requests: 2.19.1
scikit-learn: 0.20.2
scipy: 1.1.0
seaborn: 0.9.0
All 11 python packages installed

You are running Perl v 5.028001. Now checking perl modules...
Bio::Perl: 1.007002
Carp: 1.50
Clone: 0.41
DBD::SQLite: 1.62
DBD::mysql: 4.049
DBI: 1.642
DB_File: 1.84
Data::Dumper: 2.173
File::Basename: 2.85
File::Which: 1.23
Getopt::Long: 2.5
Hash::Merge: 0.300
JSON: 4.00
LWP::UserAgent: 6.36
Logger::Simple: 2.0
POSIX: 1.84
Parallel::ForkManager: 2.02
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.08
Text::Soundex: 3.05
Thread::Queue: 3.13
Tie::File: 1.02
URI::Escape: 3.31
YAML: 1.27
threads: 2.22
threads::shared: 1.59
All 27 Perl modules installed

Checking external dependencies...
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/bin/ete3", line 7, in <module>
    from ete3.tools.ete import main
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/site-packages/ete3/tools/ete.py", line 55, in <module>
    from . import (ete_split, ete_expand, ete_annotate, ete_ncbiquery, ete_view,
  File "/home/linuxbrew/.linuxbrew/Cellar/python@2/2.7.16/lib/python2.7/site-packages/ete3/tools/ete_view.py", line 48, in <module>
    from .. import (Tree, PhyloTree, TextFace, RectFace, faces, TreeStyle, CircleFace, AttrFace,
ImportError: cannot import name TextFace
RepeatMasker: RepeatMasker 4.0.7
RepeatModeler: RepeatModeler 1.0.8
Trinity: 2.8.3
augustus: 3.3.2
bamtools: bamtools 2.5.1
bedtools: bedtools v2.28.0
blat: BLAT v36
diamond: diamond 0.9.24
emapper.py: emapper-1.0.3
exonerate: exonerate 2.2.0
fasta: no way to determine
gmap: 2018-07-04
gmes_petap.pl: 4.38
hisat2: 2.1.0
hmmscan: HMMER 3.2.1 (June 2018)
hmmsearch: HMMER 3.2.1 (June 2018)
java: 9.0.4
kallisto: 0.45.1
mafft: v7.407 (2018/Jul/23)
makeblastdb: makeblastdb 2.8.1+
minimap2: 2.16-r922
nucmer: 3.1
pslCDnaFilter: no way to determine
rmblastn: rmblastn 2.2.27+
samtools: samtools 1.9
stringtie: 1.3.4d
tRNAscan-SE: 2.0 (December 2017)
tbl2asn: unknown, likely 25.3
tblastn: tblastn 2.8.1+
trimal: trimAl v1.4.rev15 build[2013-12-17]
    ERROR: CodingQuarry not installed
    ERROR: ete3 not installed
Checking Environmental Variables...
$FUNANNOTATE_DB=/home/seq/fundb
$PASAHOME=/home/seq/PASApipeline-v2.3.3
$TRINITYHOME=/home/linuxbrew/.linuxbrew/opt/trinity
$EVM_HOME=/home/linuxbrew/.linuxbrew/opt/evidencemodeler
$AUGUSTUS_CONFIG_PATH=/home/linuxbrew/.linuxbrew/opt/augustus/config
$GENEMARK_PATH=/home/seq/gmes_petap
$BAMTOOLS_PATH=/home/linuxbrew/.linuxbrew/opt/bamtools
All 7 environmental variables are set
mswyers commented 5 years ago

The reason is this is a problem is because for some reason every time I try to run $funannotate annotate I get this error: [09:49 AM]: Database files not found in /home/seq/fundb, run funannotate database and/or funannotate setup

So I figured that re-installing all of the databases would fix this. Not sure why it's saying database files are not found, because my output of funannotate database is normal:

$ funannotate database

Funannotate Databases currently installed:

  Database          Type        Version      Date         Num_Records   Md5checksum                     
  pfam              hmmer3      32.0         2018-08            17929   de7496fad69c1040fd74db1cb5eef0fc
  gene2product      text        1.32         2019-03-27         26175   3278dd8de7513e0ba2d6f3f3f1e0b1fd
  interpro          xml         73.0         2019-03-21         36702   9e1b0aeb614451d08ca4e3688b844c64
  dbCAN             hmmer3      6.0          2017-09-12           585   3cb06f6f93c72a56c9fa12a6294b41d5
  busco_outgroups   outgroups   1.0          2018-07-31             8   6795b1d4545850a4226829c7ae8ef058
  merops            diamond     12.0         2017-10-04          4968   d923f0177c6d27c3d2886c705347adc0
  mibig             diamond     1.4          2019-04-08         31023   118f2c11edde36c81bdea030a0228492
  uniprot           diamond     2019_02      2019-02-13        559228   7b42b43b6a9cc28f2424d2fa73ae4852
  go                text        2019-03-19   2019-03-19         47362   00575e6917279fc76cfd1b076406d1ac
  repeats           diamond     1.0          2019-04-08         11950   4e8cafc3eea47ec7ba505bb1e3465d21

To update a database type:
    funannotate setup -i DBNAME -d /home/seq/fundb --force
nextgenusfs commented 5 years ago

Same as in #277. Upgrade to 1.5.3.

mswyers commented 5 years ago

@nextgenusfs

I originally installed using the homebrew-mediated installation, which at the moment only upgrades to version 1.5.2. Do I need to use the manual install method to get the latest version instead, and how do I get that to be the recognized version instead of the linuxbrew version?

mswyers commented 5 years ago

Looking at the funannotate homebrew tap, the version listed there is still 1.5.2.