search

nextgenusfs / funannotate

Eukaryotic Genome Annotation Pipeline
http://funannotate.readthedocs.io
BSD 2-Clause "Simplified" License
301 stars 82 forks source link

Conda installation problem #365

Closed emmannaemeka closed 3 years ago

emmannaemeka commented 4 years ago

Are you using the latest release? If you are not using the latest release of funannotate, please upgrade, if bug persists then report here.

Describe the bug A clear and concise description of what the bug is.

What command did you issue? Copy/paste the command used. conda create -n funannotate funannotate Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: - Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Package stringtie conflicts for: funannotate -> stringtie Package perl-json conflicts for: funannotate -> perl-json Package snap conflicts for: funannotate -> snap Package tantan conflicts for: funannotate -> tantan Package trimmomatic conflicts for: funannotate -> trimmomatic Package natsort conflicts for: funannotate -> natsort Package bedtools conflicts for: funannotate -> bedtools Package diamond conflicts for: funannotate -> diamond[version='>0.9,<=0.9.24'] Package proteinortho conflicts for: funannotate -> proteinortho[version='>=6.0.9'] Package seaborn conflicts for: funannotate -> seaborn Package matplotlib conflicts for: funannotate -> matplotlib Package hmmer conflicts for: funannotate -> hmmer Package scikit-learn conflicts for: funannotate -> scikit-learn Package biopython conflicts for: funannotate -> biopython Package ucsc-pslcdnafilter conflicts for: funannotate -> ucsc-pslcdnafilter Package perl conflicts for: funannotate -> perl Package raxml conflicts for: funannotate -> raxml Package hisat2 conflicts for: funannotate -> hisat2 Package perl-dbd-mysql conflicts for: funannotate -> perl-dbd-mysql Package pandas conflicts for: funannotate -> pandas Package pasa conflicts for: funannotate -> pasa[version='>=2.4.1'] Package kallisto conflicts for: funannotate -> kallisto Package numpy conflicts for: funannotate -> numpy Package mafft conflicts for: funannotate -> mafft[version='>=7'] Package goatools conflicts for: funannotate -> goatools Package trimal conflicts for: funannotate -> trimal Package perl-parallel-forkmanager conflicts for: funannotate -> perl-parallel-forkmanager Package blast conflicts for: funannotate -> blast==2.2.31 Package salmon conflicts for: funannotate -> salmon[version='>=0.9'] Package glimmerhmm conflicts for: funannotate -> glimmerhmm Package perl-hash-merge conflicts for: funannotate -> perl-hash-merge Package evidencemodeler conflicts for: funannotate -> evidencemodeler Package exonerate conflicts for: funannotate -> exonerate Package psutil conflicts for: funannotate -> psutil Package scipy conflicts for: funannotate -> scipy Package python conflicts for: funannotate -> python[version='>=2.7,<2.8.0a0'] Package perl-bioperl conflicts for: funannotate -> perl-bioperl[version='>1.7'] Package perl-scalar-util-numeric conflicts for: funannotate -> perl-scalar-util-numeric Package perl-clone conflicts for: funannotate -> perl-clone Package r-base conflicts for: funannotate -> r-base[version='>=3.4.1'] Package tbl2asn conflicts for: funannotate -> tbl2asn Package minimap2 conflicts for: funannotate -> minimap2 Package perl-text-soundex conflicts for: funannotate -> perl-text-soundex Package requests conflicts for: funannotate -> requests Package codingquarry conflicts for: funannotate -> codingquarry Package iqtree conflicts for: funannotate -> iqtree Package ete3 conflicts for: funannotate -> ete3 Package trnascan-se conflicts for: funannotate -> trnascan-se Package perl-soap-lite conflicts for: funannotate -> perl-soap-lite Package perl-logger-simple conflicts for: funannotate -> perl-logger-simple Logfiles Please provide relavent log files of the error.

OS/Install Information

nextgenusfs commented 4 years ago

I’m not sure, we likely need to get some help from bioconda folks. But it might be happening if you have packages installed in your base conda installation. Could test this by installing a fresh copy of conda and then try to create the environment as you did here. The recommended way is to not install anything in the conda base env.

nextgenusfs commented 4 years ago

@jolespin did you ever figure out this conda "UnsatisfiableError" error? https://github.com/conda/conda/issues/8823

Not sure what the problem is -- but I see this in many issues on many different packages. All I can say is that I've tested it on macOS running conda 4.7.12 and also centOS with 4.6.14. I know that likely isn't helpful.... but trying a different conda version (upgrade or downgrade) would be the next thing I'd try.

jolespin commented 4 years ago

It's getting there. I tried installing a new environment with your conda package. Here's what I got so far: 

conda create --name funannotate2_env  -c bioconda funannotate -y
nano /usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/etc/conda/activate.d/funannotate_activate.sh

Here's my activation script: 

echo "----------------------------------"
echo "Activating Funannotate Environment"
echo "----------------------------------"

# Paths
PACKAGES=/usr/local/devel/ANNOTATION/jespinoz/Packages
OPT=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/opt
BIN=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin
HOME_DIRECTORY=~
eval HOME_DIRECTORY=$HOME_DIRECTORY

# Exports
export GENEMARK_PATH=${PACKAGES}/GeneMark-ES_ET-4.46/gm_et_linux_64
export FUNANNOTATE_DB=/usr/local/scratch/METAGENOMICS/jespinoz/db/funannotate_db

# GeneMark-ES_ET License
HOME_DIRECTORY=~
eval HOME_DIRECTORY=$HOME_DIRECTORY
GM_KEY="${HOME_DIRECTORY}/.gm_key"
if [ -f "$GM_KEY" ]; then
    echo -e "GeneMark-ES_ET-4.46 license already exists: ${GM_KEY}\nPlease delete the file and reactivate environment if you to overwrite the license file."
else
    echo "GeneMark-ES_ET-4.46 license does not exist.  Copying license into ${GM_KEY}"
    cp ${PACKAGES}/GeneMark-ES_ET-4.46/gm_et_linux_64/gm_key_64 ${GM_KEY}
fi

conda activate funannotate2_env funannotate check --show-versions 

(funannotate2_env) -bash-4.1$ funannotate check --show-versions
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
Checking dependencies for 1.7.2
-------------------------------------------------------
You are running Python v 2.7.15. Now checking python packages...
biopython: 1.68
goatools: 0.9.9
matplotlib: 2.2.4
natsort: 6.2.0
numpy: 1.16.5
pandas: 0.24.2
psutil: 5.6.7
requests: 2.22.0
scikit-learn: 0.20.3
scipy: 1.2.1
seaborn: 0.9.0
All 11 python packages installed

You are running Perl v 5.026002. Now checking perl modules...
Bio::Perl: 1.007002
Carp: 1.38
Clone: 0.42
DBD::SQLite: 1.64
DBD::mysql: 4.046
DBI: 1.642
DB_File: 1.852
Data::Dumper: 2.173
File::Basename: 2.85
File::Which: 1.23
Getopt::Long: 2.5
Hash::Merge: 0.300
JSON: 4.02
LWP::UserAgent: 6.39
Logger::Simple: 2.0
POSIX: 1.76
Parallel::ForkManager: 2.02
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.15
Text::Soundex: 3.05
Thread::Queue: 3.12
Tie::File: 1.02
URI::Escape: 3.31
YAML: 1.29
threads: 2.15
threads::shared: 1.56
All 27 Perl modules installed

Checking Environmental Variables...
$FUNANNOTATE_DB=/usr/local/scratch/METAGENOMICS/jespinoz/db/funannotate_db
$PASAHOME=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/opt/pasa-2.4.1
$TRINITY_HOME=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/opt/trinity-2.8.5
$EVM_HOME=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/opt/evidencemodeler-1.1.1
$AUGUSTUS_CONFIG_PATH=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/config/
$GENEMARK_PATH=/usr/local/devel/ANNOTATION/jespinoz/Packages/GeneMark-ES_ET-4.46/gm_et_linux_64
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
PASA: 2.4.1
CodingQuarry: 2.0
Trinity: 2.8.5
augustus: 3.3.3
bamtools: bamtools 2.5.1
bedtools: bedtools v2.29.2
blat: BLAT v36
diamond: 0.9.21
ete3: 3.1.1
exonerate: exonerate 2.4.0
fasta: no way to determine
glimmerhmm: 3.0.4
gmap: 2017-11-15
hisat2: 2.1.0
hmmscan: HMMER 3.2.1 (June 2018)
hmmsearch: HMMER 3.2.1 (June 2018)
java: 11.0.1-internal
kallisto: 0.46.1
mafft: v7.455 (2019/Dec/7)
makeblastdb: makeblastdb 2.2.31+
minimap2: 2.17-r941
proteinortho: 6.0.12
pslCDnaFilter: no way to determine
salmon: salmon 0.15.0
samtools: samtools 1.9
signalp: 4.1
snap: 2006-07-28
stringtie: 2.0
tRNAscan-SE: 2.0.5 (October 2019)
tantan: tantan 13
tbl2asn: no way to determine, likely 25.X
tblastn: tblastn 2.2.31+
trimal: trimAl v1.4.rev15 build[2013-12-17]
trimmomatic: 0.39
    ERROR: emapper.py not installed
    ERROR: gmes_petap.pl not installed

It's having trouble finding my gmes_petap.pl for some reason b/c it's in the specified directory:

(funannotate2_env) -bash-4.1$ echo $GENEMARK_PATH
/usr/local/devel/ANNOTATION/jespinoz/Packages/GeneMark-ES_ET-4.46/gm_et_linux_64
(funannotate2_env) -bash-4.1$ ls  $GENEMARK_PATH
bet_to_gff.pl                   cmd_rename_shebang.sh     gmes_petap.pl  hmm_to_gtf.pl        other                predict_genes.pl          rescale_gff.pl            star_to_gff.pl
bp_seq_select.pl                gc_distr.pl               gmhmme3        INSTALL              parse_by_introns.pl  prior.bp                  rnaseq_introns_to_gff.pl  statistics
build_mod.pl                    get_sequence_from_GTF.pl  gm_key_64      lib                  parse_ET.pl          probuild                  run_es.pl                 verify_evidence_gmhmm.pl
calc_introns_from_gtf.pl        Gibbs3                    heu_dir        LICENSE              parse_gibbs.pl       README.GeneMark-ES-suite  run_hmm_pbs.pl            wbp.cfg
change_path_in_perl_scripts.pl  gmes.cfg                  histogram.pl   make_nt_freq_mat.pl  parse_set.pl         reformat_gff.pl           scan_for_bp.pl
jolespin commented 4 years ago

Here is what happens when I run the test command: 

(funannotate2_env) -bash-4.1$ funannotate test -t all --cpus 4
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
#########################################################
Running `funannotate clean` unit testing: minimap2 mediated assembly duplications
Downloading: https://osf.io/8pjbe/download?version=1 Bytes: 252076
CMD: funannotate clean -i test.clean.fa -o test.exhaustive.fa --exhaustive
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-----------------------------------------------
6 input contigs, 6 larger than 500 bp, N50 is 427,039 bp
Checking duplication of 6 contigs
-----------------------------------------------
scaffold_91 appears duplicated: 100% identity over 100% of the contig. contig length: 8858
scaffold_73 appears duplicated: 100% identity over 100% of the contig. contig length: 15153
scaffold_27 appears duplicated: 100% identity over 100% of the contig. contig length: 427039
-----------------------------------------------
6 input contigs; 6 larger than 500 bp; 3 duplicated; 3 written to file
#########################################################
SUCCESS: `funannotate clean` test complete.
#########################################################

#########################################################
Running `funannotate mask` unit testing: RepeatModeler --> RepeatMasker
Downloading: https://osf.io/hbryz/download?version=1 Bytes: 375687
CMD: funannotate mask -i test.fa -o test.masked.fa --cpus 4
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[01:01 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[01:01 AM]: Running funanotate v1.7.2
[01:01 AM]: Soft-masking simple repeats with tantan
[01:01 AM]: Repeat soft-masking finished:
Masked genome: /local/ifs3_scratch/METAGENOMICS/jespinoz/TMPDIR/test-mask_88801/test.masked.fa
num scaffolds: 2
assembly size: 1,216,048 bp
masked repeats: 50,965 bp (4.19%)
-------------------------------------------------------
#########################################################
SUCCESS: `funannotate mask` test complete.
#########################################################

#########################################################
Running `funannotate predict` unit testing
Downloading: https://osf.io/te2pf/download?version=1 Bytes: 1489808
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 4 --species Awesome testicus
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[01:01 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[01:01 AM]: Running funannotate v1.7.2
[01:01 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[01:01 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[01:01 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-lfKHHb/.nfs000000014aedd9420000032e'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-pOOARx/.nfs000000014ba01efc0000032f'
[01:02 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[01:02 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[01:02 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[01:02 AM]: Exonerate finished: found 1,347 alignments
[01:02 AM]: Running GeneMark-ES on assembly
[01:06 AM]: 1,540 predictions from GeneMark
[01:06 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[01:20 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[01:20 AM]: Running EVM commands with 3 CPUs
[01:21 AM]: Converting to GFF3 and collecting all EVM results
[01:21 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[01:21 AM]: 387 BUSCO predictions validated
[01:21 AM]: Running Augustus gene prediction using saccharomyces parameters
[01:24 AM]: 1,607 predictions from Augustus
[01:24 AM]: Pulling out high quality Augustus predictions
[01:24 AM]: Found 376 high quality predictions from Augustus (>90% exon evidence)
[01:24 AM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[01:24 AM]: 0 predictions from SNAP
[01:24 AM]: SNAP prediction failed, moving on without result
[01:24 AM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[01:26 AM]: 1,784 predictions from GlimmerHMM
[01:26 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1449
  Augustus HiQ   2        377
  GeneMark       1        1540
  GlimmerHMM     1        1784
  Total          -        5150
[01:26 AM]: Running EVM commands with 3 CPUs
[01:28 AM]: Converting to GFF3 and collecting all EVM results
[01:28 AM]: 1,718 total gene models from EVM
[01:28 AM]: Generating protein fasta files from 1,718 EVM models
[01:28 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[01:28 AM]: Found 114 gene models to remove: 0 too short; 0 span gaps; 155 transposable elements
[01:28 AM]: 1,604 gene models remaining
[01:28 AM]: Predicting tRNAs
[01:29 AM]: 112 tRNAscan models are valid (non-overlapping)
[01:29 AM]: Generating GenBank tbl annotation file
[01:29 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[01:29 AM]: Collecting final annotation files for 1,716 total gene models
[01:29 AM]: Funannotate predict is finished, output files are in the annotate/predict_results folder
[01:29 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[01:29 AM]: Training parameters file saved: annotate/predict_results/saccharomyces.parameters.json
[01:29 AM]: Add species parameters to database:

  funannotate species -s saccharomyces -a annotate/predict_results/saccharomyces.parameters.json

#########################################################
SUCCESS: `funannotate predict` test complete.
#########################################################

#########################################################
Running `funannotate predict` BUSCO-mediated training unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --cpus 4 --species Awesome busco
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[01:29 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[01:29 AM]: Running funannotate v1.7.2
[01:29 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[01:29 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[01:29 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-vEs3Bl/.nfs000000014aede264000003c1'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-BwqwSt/.nfs000000014ba02b8d000003c2'
[01:29 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[01:29 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[01:29 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[01:30 AM]: Exonerate finished: found 1,347 alignments
[01:30 AM]: Running GeneMark-ES on assembly
[01:33 AM]: 1,536 predictions from GeneMark
[01:33 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[01:46 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[01:47 AM]: Running EVM commands with 3 CPUs
[01:47 AM]: Converting to GFF3 and collecting all EVM results
[01:47 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[01:47 AM]: 387 BUSCO predictions validated
[01:47 AM]: Training Augustus using BUSCO gene models
[01:47 AM]: Augustus initial training results:
  Feature       Specificity   Sensitivity
  nucleotides   99.2%         90.1%
  exons         84.9%         65.2%
  genes         87.0%         63.5%
[01:47 AM]: Running Augustus gene prediction using awesome_busco parameters
[01:49 AM]: 1,407 predictions from Augustus
[01:49 AM]: Pulling out high quality Augustus predictions
[01:49 AM]: Found 321 high quality predictions from Augustus (>90% exon evidence)
[01:49 AM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[01:49 AM]: 0 predictions from SNAP
[01:49 AM]: SNAP prediction failed, moving on without result
[01:49 AM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[01:50 AM]: 1,780 predictions from GlimmerHMM
[01:50 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1086
  Augustus HiQ   2        321
  GeneMark       1        1536
  GlimmerHMM     1        1780
  Total          -        4723
[01:50 AM]: Running EVM commands with 3 CPUs
[01:52 AM]: Converting to GFF3 and collecting all EVM results
[01:52 AM]: 1,686 total gene models from EVM
[01:52 AM]: Generating protein fasta files from 1,686 EVM models
[01:52 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[01:52 AM]: Found 96 gene models to remove: 0 too short; 0 span gaps; 120 transposable elements
[01:52 AM]: 1,590 gene models remaining
[01:52 AM]: Predicting tRNAs
[01:53 AM]: 111 tRNAscan models are valid (non-overlapping)
[01:53 AM]: Generating GenBank tbl annotation file
[01:53 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[01:53 AM]: Collecting final annotation files for 1,701 total gene models
[01:53 AM]: Funannotate predict is finished, output files are in the annotate/predict_results folder
[01:53 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[01:53 AM]: Training parameters file saved: annotate/predict_results/awesome_busco.parameters.json
[01:53 AM]: Add species parameters to database:

  funannotate species -s awesome_busco -a annotate/predict_results/awesome_busco.parameters.json

#########################################################
SUCCESS: `funannotate predict` BUSCO-mediated training test complete.
#########################################################
Now running predict using all pre-trained ab-initio predictors
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate2 --cpus 4 --species Awesome busco -p annotate/predict_results/awesome_busco.parameters.json
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[01:53 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[01:53 AM]: Running funannotate v1.7.2
[01:53 AM]: Ab initio training parameters file passed: annotate/predict_results/awesome_busco.parameters.json
[01:53 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained
  genemark     pretrained
  glimmerhmm   pretrained
  snap         busco
[01:53 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[01:53 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-gPFxa1/.nfs000000014b79947e0000044a'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-rnOcj5/.nfs000000014bb209ae0000044b'
[01:53 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[01:53 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[01:53 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[01:54 AM]: Exonerate finished: found 1,347 alignments
[01:54 AM]: Running GeneMark-ES on assembly
[01:57 AM]: 1,541 predictions from GeneMark
[01:57 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[02:10 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[02:10 AM]: Running EVM commands with 3 CPUs
[02:11 AM]: Converting to GFF3 and collecting all EVM results
[02:11 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[02:11 AM]: 387 BUSCO predictions validated
[02:11 AM]: Running Augustus gene prediction using awesome_busco parameters
[02:12 AM]: 1,407 predictions from Augustus
[02:13 AM]: Pulling out high quality Augustus predictions
[02:13 AM]: Found 321 high quality predictions from Augustus (>90% exon evidence)
[02:13 AM]: Running SNAP gene prediction, using training data: annotate2/predict_misc/busco.final.gff3
[02:13 AM]: 0 predictions from SNAP
[02:13 AM]: SNAP prediction failed, moving on without result
[02:13 AM]: Running GlimmerHMM gene prediction, using pretrained HMM profile
[02:13 AM]: 1,780 predictions from GlimmerHMM
[02:13 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1086
  Augustus HiQ   2        321
  GeneMark       1        1541
  GlimmerHMM     1        1780
  Total          -        4728
[02:13 AM]: Running EVM commands with 3 CPUs
[02:15 AM]: Converting to GFF3 and collecting all EVM results
[02:15 AM]: 1,685 total gene models from EVM
[02:15 AM]: Generating protein fasta files from 1,685 EVM models
[02:15 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[02:15 AM]: Found 96 gene models to remove: 0 too short; 0 span gaps; 120 transposable elements
[02:15 AM]: 1,589 gene models remaining
[02:15 AM]: Predicting tRNAs
[02:16 AM]: 111 tRNAscan models are valid (non-overlapping)
[02:16 AM]: Generating GenBank tbl annotation file
[02:16 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[02:16 AM]: Collecting final annotation files for 1,700 total gene models
[02:16 AM]: Funannotate predict is finished, output files are in the annotate2/predict_results folder
[02:16 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate2 -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate2 -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate2 --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[02:16 AM]: Training parameters file saved: annotate2/predict_results/awesome_busco.parameters.json
[02:16 AM]: Add species parameters to database:

  funannotate species -s awesome_busco -a annotate2/predict_results/awesome_busco.parameters.json

#########################################################
SUCCESS: `funannotate predict` using existing parameters test complete.
#########################################################

#########################################################
Running funannotate RNA-seq training/prediction unit testing
Downloading: https://osf.io/t7j83/download?version=1 Bytes: 542753017
CMD: funannotate train -i test.softmasked.fa --single rna-seq.illumina.fastq.gz --nanopore_mrna rna-seq.nanopore.fastq.gz -o rna-seq --cpus 4 --jaccard_clip --species Awesome rna
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[02:17 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[02:17 AM]: Running 1.7.2
[02:17 AM]: Adapter and Quality trimming SE reads with Trimmomatic
[02:18 AM]: Running read normalization with Trinity
[02:22 AM]: Processing long reads: converting to fasta and running SeqClean
[02:24 AM]: Building Hisat2 genome index
[02:24 AM]: Aligning reads to genome using Hisat2
[02:24 AM]: Running genome-guided Trinity, logfile: rna-seq/training/Trinity-gg.log
[02:24 AM]: Clustering of reads from BAM and preparing assembly commands
[02:28 AM]: Assembling 1,601 Trinity clusters using 3 CPUs
[03:03 AM]: 1,450 transcripts derived from Trinity
[03:03 AM]: Running StringTie on Hisat2 coordsorted BAM
[03:03 AM]: Removing poly-A sequences from trinity transcripts using seqclean
[03:03 AM]: Aligning long reads to genome with minimap2
[03:03 AM]: Finding long-reads not represented in Trinity assemblies
[03:03 AM]: Adding 3,496 unique long-reads to Trinity assemblies
[03:03 AM]: Merging BAM files: rna-seq/training/nano_mRNA.coordSorted.bam, rna-seq/training/trinity.alignments.bam
[03:03 AM]: Converting transcript alignments to GFF3 format
[03:04 AM]: Converting Trinity transcript alignments to GFF3 format
[03:04 AM]: Running PASA alignment step using 4,946 transcripts
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 657, in <module>
    main()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 647, in main
    mod.main(arguments)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/train.py", line 1059, in main
    pasa_db=args.pasa_db, pasa_alignment_overlap=args.pasa_alignment_overlap, aligners=args.aligners)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/train.py", line 342, in runPASAtrain
    with open(os.path.join(folder, pasaDBname+'.pasa_assemblies_described.txt'), 'r') as description:
IOError: [Errno 2] No such file or directory: 'rna-seq/training/pasa/Awesome_rna.pasa_assemblies_described.txt'
#########################################################
Now running `funannotate predict` using RNA-seq training data
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o rna-seq --cpus 4 --min_training_models 150 --species Awesome rna
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[03:09 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[03:09 AM]: Running funannotate v1.7.2
[03:09 AM]: Found training files, will re-use these files:
  --stringtie rna-seq/training/funannotate_train.stringtie.gtf
[03:09 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[03:09 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[03:09 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-gWqLtQ/.nfs000000014b86fa200000166a'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-OMZglD/.nfs000000014b86fa210000166b'
[03:09 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[03:09 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[03:10 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[03:10 AM]: Exonerate finished: found 1,347 alignments
[03:11 AM]: Running GeneMark-ES on assembly
[03:14 AM]: 1,536 predictions from GeneMark
[03:14 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[03:27 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[03:27 AM]: Running EVM commands with 3 CPUs
[03:28 AM]: Converting to GFF3 and collecting all EVM results
[03:28 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[03:28 AM]: 387 BUSCO predictions validated
[03:28 AM]: Training Augustus using BUSCO gene models
[03:28 AM]: Augustus initial training results:
  Feature       Specificity   Sensitivity
  nucleotides   99.2%         90.1%
  exons         84.9%         65.2%
  genes         87.0%         63.5%
[03:28 AM]: Running Augustus gene prediction using awesome_rna parameters
[03:30 AM]: 1,407 predictions from Augustus
[03:30 AM]: Pulling out high quality Augustus predictions
[03:30 AM]: Found 321 high quality predictions from Augustus (>90% exon evidence)
[03:30 AM]: Running SNAP gene prediction, using training data: rna-seq/predict_misc/busco.final.gff3
[03:30 AM]: 0 predictions from SNAP
[03:30 AM]: SNAP prediction failed, moving on without result
[03:30 AM]: Running GlimmerHMM gene prediction, using training data: rna-seq/predict_misc/busco.final.gff3
[03:31 AM]: 1,786 predictions from GlimmerHMM
[03:31 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1086
  Augustus HiQ   2        321
  GeneMark       1        1536
  GlimmerHMM     1        1786
  Total          -        4729
[03:31 AM]: Running EVM commands with 3 CPUs
[03:33 AM]: Converting to GFF3 and collecting all EVM results
[03:33 AM]: 1,688 total gene models from EVM
[03:33 AM]: Generating protein fasta files from 1,688 EVM models
[03:33 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[03:34 AM]: Found 96 gene models to remove: 0 too short; 0 span gaps; 120 transposable elements
[03:34 AM]: 1,592 gene models remaining
[03:34 AM]: Predicting tRNAs
[03:34 AM]: 111 tRNAscan models are valid (non-overlapping)
[03:34 AM]: Generating GenBank tbl annotation file
[03:34 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[03:34 AM]: Collecting final annotation files for 1,703 total gene models
[03:34 AM]: Funannotate predict is finished, output files are in the rna-seq/predict_results folder
[03:34 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i rna-seq -m docker -c 4

Run antiSMASH:
funannotate remote -i rna-seq -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i rna-seq --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[03:34 AM]: Training parameters file saved: rna-seq/predict_results/awesome_rna.parameters.json
[03:34 AM]: Add species parameters to database:

  funannotate species -s awesome_rna -a rna-seq/predict_results/awesome_rna.parameters.json

#########################################################
Now running `funannotate update` to run PASA-mediated UTR addition and multiple transcripts
CMD: funannotate update -i rna-seq --cpus 4
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[03:34 AM]: OS: linux2, 4 cores, ~ 8 GB RAM. Python: 2.7.15
[03:34 AM]: Running 1.7.2
[03:34 AM]: No NCBI SBT file given, will use default, for NCBI submissions pass one here '--sbt'
[03:34 AM]: Found relevant files in rna-seq/training, will re-use them:
    Single reads: rna-seq/training/single.fq.gz
    Single Q-trimmed reads: rna-seq/training/trimmomatic/trimmed_single.fastq.gz
    Single normalized reads: rna-seq/training/normalize/single.norm.fq
    PASA config file: rna-seq/training/pasa/alignAssembly.txt
    StringTie GTF: rna-seq/training/funannotate_train.stringtie.gtf
[03:34 AM]: Reannotating Awesome rna, NCBI accession: None
[03:34 AM]: Previous annotation consists of: 1,592 protein coding gene models and 111 non-coding gene models
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 657, in <module>
    main()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 647, in main
    mod.main(arguments)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/update.py", line 2050, in main
    if lib.checkannotations(longReadClean):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/library.py", line 883, in checkannotations
    if os.path.isfile(input):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/genericpath.py", line 37, in isfile
    st = os.stat(path)
TypeError: coercing to Unicode: need string or buffer, NoneType found
#########################################################
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 657, in <module>
    main()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/bin/funannotate", line 647, in main
    mod.main(arguments)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/test.py", line 393, in main
    runRNAseqTest(args)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/test.py", line 345, in runRNAseqTest
    tmpdir, 'rna-seq', 'update_results', 'Awesome_rna.gff3')) <= 1830
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/funannotate/test.py", line 40, in countGFFgenes
    with open(input, 'rU') as f:
IOError: [Errno 2] No such file or directory: 'test-rna_seq_88801/rna-seq/update_results/Awesome_rna.gff3'
(funannotate2_env) -bash-4.1$
nextgenusfs commented 4 years ago

Well that's getting better.... seems like PASA is dying. Any useful information in the pasa logs (would be nested in the training folder of this test run -- since it failed, the tmpdir should still be there)?

jolespin commented 4 years ago

Not sure why but it worked this time ?!

I couldnt find my last run so i started over

(base) -bash-4.1$ conda activate funannotate2_env
----------------------------------
Activating Funannotate Environment
----------------------------------
GeneMark-ES_ET-4.46 license already exists: /home/jespinoz/.gm_key
Please delete the file and reactivate environment if you to overwrite the license file.
(funannotate2_env) -bash-4.1$ funannotate test -t all --cpus 4
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
#########################################################
Running `funannotate clean` unit testing: minimap2 mediated assembly duplications
Downloading: https://osf.io/8pjbe/download?version=1 Bytes: 252076
CMD: funannotate clean -i test.clean.fa -o test.exhaustive.fa --exhaustive
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-----------------------------------------------
6 input contigs, 6 larger than 500 bp, N50 is 427,039 bp
Checking duplication of 6 contigs
-----------------------------------------------
scaffold_91 appears duplicated: 100% identity over 100% of the contig. contig length: 8858
scaffold_73 appears duplicated: 100% identity over 100% of the contig. contig length: 15153
scaffold_27 appears duplicated: 100% identity over 100% of the contig. contig length: 427039
-----------------------------------------------
6 input contigs; 6 larger than 500 bp; 3 duplicated; 3 written to file
#########################################################
SUCCESS: `funannotate clean` test complete.
#########################################################

#########################################################
Running `funannotate mask` unit testing: RepeatModeler --> RepeatMasker
Downloading: https://osf.io/hbryz/download?version=1 Bytes: 375687
CMD: funannotate mask -i test.fa -o test.masked.fa --cpus 4
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[03:01 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[03:01 AM]: Running funanotate v1.7.2
[03:01 AM]: Soft-masking simple repeats with tantan
[03:01 AM]: Repeat soft-masking finished:
Masked genome: /local/ifs3_scratch/METAGENOMICS/jespinoz/Testing/test-mask_88701/test.masked.fa
num scaffolds: 2
assembly size: 1,216,048 bp
masked repeats: 50,965 bp (4.19%)
-------------------------------------------------------
#########################################################
SUCCESS: `funannotate mask` test complete.
#########################################################

#########################################################
Running `funannotate predict` unit testing
Downloading: https://osf.io/te2pf/download?version=1 Bytes: 1489808
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 4 --species Awesome testicus
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[03:01 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[03:01 AM]: Running funannotate v1.7.2
[03:01 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[03:01 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[03:01 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-M3_1kG/.nfs000000014cc0b643000005e3'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-WJbKzk/.nfs000000014cb781f6000005e4'
[03:01 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[03:01 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[03:01 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[03:02 AM]: Exonerate finished: found 1,347 alignments
[03:03 AM]: Running GeneMark-ES on assembly
[03:08 AM]: 1,547 predictions from GeneMark
[03:08 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[03:32 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[03:33 AM]: Running EVM commands with 3 CPUs
[03:33 AM]: Converting to GFF3 and collecting all EVM results
[03:33 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[03:34 AM]: 387 BUSCO predictions validated
[03:34 AM]: Running Augustus gene prediction using saccharomyces parameters
[03:39 AM]: 1,607 predictions from Augustus
[03:39 AM]: Pulling out high quality Augustus predictions
[03:39 AM]: Found 376 high quality predictions from Augustus (>90% exon evidence)
[03:39 AM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[03:41 AM]: 12 predictions from SNAP
[03:41 AM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[03:43 AM]: 1,783 predictions from GlimmerHMM
[03:43 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1449
  Augustus HiQ   2        377
  GeneMark       1        1547
  GlimmerHMM     1        1783
  snap           1        12
  Total          -        5168
[03:43 AM]: Running EVM commands with 3 CPUs
[03:47 AM]: Converting to GFF3 and collecting all EVM results
[03:47 AM]: 1,718 total gene models from EVM
[03:47 AM]: Generating protein fasta files from 1,718 EVM models
[03:47 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[03:48 AM]: Found 114 gene models to remove: 0 too short; 0 span gaps; 154 transposable elements
[03:48 AM]: 1,604 gene models remaining
[03:48 AM]: Predicting tRNAs
[03:48 AM]: 112 tRNAscan models are valid (non-overlapping)
[03:48 AM]: Generating GenBank tbl annotation file
[03:48 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[03:48 AM]: Collecting final annotation files for 1,716 total gene models
[03:48 AM]: Funannotate predict is finished, output files are in the annotate/predict_results folder
[03:48 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[03:48 AM]: Training parameters file saved: annotate/predict_results/saccharomyces.parameters.json
[03:48 AM]: Add species parameters to database:

  funannotate species -s saccharomyces -a annotate/predict_results/saccharomyces.parameters.json

#########################################################
SUCCESS: `funannotate predict` test complete.
#########################################################

#########################################################
Running `funannotate predict` BUSCO-mediated training unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --cpus 4 --species Awesome busco
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[03:49 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[03:49 AM]: Running funannotate v1.7.2
[03:49 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[03:49 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[03:49 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-CyHGhX/.nfs000000014cb0c83f0000066c'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-sgC4ka/.nfs000000014cbab5c30000066d'
[03:49 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[03:49 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[03:49 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[03:50 AM]: Exonerate finished: found 1,347 alignments
[03:50 AM]: Running GeneMark-ES on assembly
[03:55 AM]: 1,535 predictions from GeneMark
[03:55 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[04:20 AM]: 373 valid BUSCO predictions found, now formatting for EVM
[04:20 AM]: Running EVM commands with 3 CPUs
[04:21 AM]: Converting to GFF3 and collecting all EVM results
[04:21 AM]: 387 total gene models from EVM, now validating with BUSCO HMM search
[04:21 AM]: 387 BUSCO predictions validated
[04:21 AM]: Training Augustus using BUSCO gene models
[04:22 AM]: Augustus initial training results:
  Feature       Specificity   Sensitivity
  nucleotides   99.2%         90.1%
  exons         84.9%         65.2%
  genes         87.0%         63.5%
[04:22 AM]: Running Augustus gene prediction using awesome_busco parameters
[04:24 AM]: 1,407 predictions from Augustus
[04:24 AM]: Pulling out high quality Augustus predictions
[04:24 AM]: Found 321 high quality predictions from Augustus (>90% exon evidence)
[04:24 AM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[04:27 AM]: 12 predictions from SNAP
[04:27 AM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[04:28 AM]: 1,783 predictions from GlimmerHMM
[04:28 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1086
  Augustus HiQ   2        321
  GeneMark       1        1535
  GlimmerHMM     1        1783
  snap           1        12
  Total          -        4737
[04:28 AM]: Running EVM commands with 3 CPUs
[04:32 AM]: Converting to GFF3 and collecting all EVM results
[04:32 AM]: 1,683 total gene models from EVM
[04:32 AM]: Generating protein fasta files from 1,683 EVM models
[04:32 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[04:33 AM]: Found 99 gene models to remove: 0 too short; 0 span gaps; 126 transposable elements
[04:33 AM]: 1,584 gene models remaining
[04:33 AM]: Predicting tRNAs
[04:33 AM]: 111 tRNAscan models are valid (non-overlapping)
[04:33 AM]: Generating GenBank tbl annotation file
[04:33 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[04:33 AM]: Collecting final annotation files for 1,695 total gene models
[04:33 AM]: Funannotate predict is finished, output files are in the annotate/predict_results folder
[04:33 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[04:33 AM]: Training parameters file saved: annotate/predict_results/awesome_busco.parameters.json
[04:33 AM]: Add species parameters to database:

  funannotate species -s awesome_busco -a annotate/predict_results/awesome_busco.parameters.json

#########################################################
SUCCESS: `funannotate predict` BUSCO-mediated training test complete.
#########################################################
Now running predict using all pre-trained ab-initio predictors
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate2 --cpus 4 --species Awesome busco -p annotate/predict_results/awesome_busco.parameters.json
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[04:33 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[04:33 AM]: Running funannotate v1.7.2
[04:33 AM]: Ab initio training parameters file passed: annotate/predict_results/awesome_busco.parameters.json
[04:33 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained
  genemark     pretrained
  glimmerhmm   pretrained
  snap         pretrained
[04:33 AM]: CodingQuarry will be skipped --> --rna_bam required for training
[04:33 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-aaw0Kg/.nfs000000014cb0d271000006f5'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-7l7eH2/.nfs000000014cc0c598000006f6'
[04:33 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[04:33 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[04:34 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[04:35 AM]: Exonerate finished: found 1,347 alignments
[04:35 AM]: Running GeneMark-ES on assembly
[04:40 AM]: 1,545 predictions from GeneMark
[04:40 AM]: Running Augustus gene prediction using awesome_busco parameters
[04:42 AM]: 1,407 predictions from Augustus
[04:42 AM]: Pulling out high quality Augustus predictions
[04:42 AM]: Found 321 high quality predictions from Augustus (>90% exon evidence)
[04:42 AM]: Running SNAP gene prediction, using pre-trained HMM profile
[04:45 AM]: 12 predictions from SNAP
[04:45 AM]: Running GlimmerHMM gene prediction, using pretrained HMM profile
[04:45 AM]: 1,783 predictions from GlimmerHMM
[04:45 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1086
  Augustus HiQ   2        321
  GeneMark       1        1545
  GlimmerHMM     1        1783
  snap           1        12
  Total          -        4747
[04:45 AM]: Running EVM commands with 3 CPUs
[04:49 AM]: Converting to GFF3 and collecting all EVM results
[04:49 AM]: 1,683 total gene models from EVM
[04:49 AM]: Generating protein fasta files from 1,683 EVM models
[04:49 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[04:50 AM]: Found 99 gene models to remove: 0 too short; 0 span gaps; 126 transposable elements
[04:50 AM]: 1,584 gene models remaining
[04:50 AM]: Predicting tRNAs
[04:50 AM]: 111 tRNAscan models are valid (non-overlapping)
[04:50 AM]: Generating GenBank tbl annotation file
[04:50 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[04:50 AM]: Collecting final annotation files for 1,695 total gene models
[04:50 AM]: Funannotate predict is finished, output files are in the annotate2/predict_results folder
[04:50 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i annotate2 -m docker -c 4

Run antiSMASH:
funannotate remote -i annotate2 -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i annotate2 --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[04:50 AM]: Training parameters file saved: annotate2/predict_results/awesome_busco.parameters.json
[04:50 AM]: Add species parameters to database:

  funannotate species -s awesome_busco -a annotate2/predict_results/awesome_busco.parameters.json

#########################################################
SUCCESS: `funannotate predict` using existing parameters test complete.
#########################################################

#########################################################
Running funannotate RNA-seq training/prediction unit testing
Downloading: https://osf.io/t7j83/download?version=1 Bytes: 542753017
CMD: funannotate train -i test.softmasked.fa --single rna-seq.illumina.fastq.gz --nanopore_mrna rna-seq.nanopore.fastq.gz -o rna-seq --cpus 4 --jaccard_clip --species Awesome rna
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[04:51 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[04:51 AM]: Running 1.7.2
[04:51 AM]: Adapter and Quality trimming SE reads with Trimmomatic
[04:53 AM]: Running read normalization with Trinity
[04:58 AM]: Processing long reads: converting to fasta and running SeqClean
[05:01 AM]: Building Hisat2 genome index
[05:01 AM]: Aligning reads to genome using Hisat2
[05:02 AM]: Running genome-guided Trinity, logfile: rna-seq/training/Trinity-gg.log
[05:02 AM]: Clustering of reads from BAM and preparing assembly commands
[05:05 AM]: Assembling 1,602 Trinity clusters using 3 CPUs
[06:02 AM]: 1,459 transcripts derived from Trinity
[06:02 AM]: Running StringTie on Hisat2 coordsorted BAM
[06:02 AM]: Removing poly-A sequences from trinity transcripts using seqclean
[06:02 AM]: Aligning long reads to genome with minimap2
[06:02 AM]: Finding long-reads not represented in Trinity assemblies
[06:03 AM]: Adding 3,493 unique long-reads to Trinity assemblies
[06:03 AM]: Merging BAM files: rna-seq/training/nano_mRNA.coordSorted.bam, rna-seq/training/trinity.alignments.bam
[06:03 AM]: Converting transcript alignments to GFF3 format
[06:04 AM]: Converting Trinity transcript alignments to GFF3 format
[06:04 AM]: Running PASA alignment step using 4,952 transcripts
[06:13 AM]: PASA assigned 782 transcripts to 780 loci (genes)
[06:13 AM]: Getting PASA models for training with TransDecoder
[06:15 AM]: PASA finished. PASAweb accessible via: localhost:port/cgi-bin/index.cgi?db=/local/ifs3_scratch/METAGENOMICS/jespinoz/Testing/test-rna_seq_88701/rna-seq/training/pasa/Awesome_rna
[06:15 AM]: Using Kallisto TPM data to determine which PASA gene models to select at each locus
[06:15 AM]: Building Kallisto index
[06:15 AM]: Mapping reads using pseudoalignment in Kallisto
[06:15 AM]: Parsing expression value results. Keeping best transcript at each locus.
[06:16 AM]: Wrote 580 PASA gene models
[06:16 AM]: PASA database name: Awesome_rna
[06:16 AM]: Trinity/PASA has completed, you are now ready to run funanotate predict, for example:

  funannotate predict -i test.softmasked.fa \
            -o rna-seq -s "Awesome rna" --cpus 4

-------------------------------------------------------
#########################################################
Now running `funannotate predict` using RNA-seq training data
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o rna-seq --cpus 4 --min_training_models 150 --species Awesome rna
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[06:16 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[06:16 AM]: Running funannotate v1.7.2
[06:16 AM]: Found training files, will re-use these files:
  --rna_bam rna-seq/training/funannotate_train.coordSorted.bam
  --pasa_gff rna-seq/training/funannotate_train.pasa.gff3
  --stringtie rna-seq/training/funannotate_train.stringtie.gtf
  --transcript_alignments rna-seq/training/funannotate_train.transcripts.gff3
[06:16 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program        Training-Method
  augustus       pasa
  codingquarry   rna-bam
  genemark       selftraining
  glimmerhmm     pasa
  snap           pasa
[06:16 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-Mj6KNs/.nfs000000014cca62a700001913'
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 277, in _run_finalizers
    finalizer()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 266, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/shutil.py", line 264, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/usr/local/scratch/METAGENOMICS/jespinoz/TMPDIR/pymp-sjnuF6/.nfs000000014cbae35c00001914'
[06:16 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[06:16 AM]: Parsed 47,599 transcript alignments from: rna-seq/training/funannotate_train.transcripts.gff3
[06:16 AM]: Creating transcript EVM alignments and Augustus transcripts hintsfile
[06:16 AM]: Extracting hints from RNA-seq BAM file using bam2hints
[06:16 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[06:16 AM]: Found 1,774 preliminary alignments --> aligning with exonerate
[06:18 AM]: Exonerate finished: found 1,347 alignments
[06:18 AM]: Running GeneMark-ES on assembly
[06:23 AM]: 1,533 predictions from GeneMark
[06:23 AM]: Filtering PASA data for suitable training set
[06:23 AM]: 540 of 580 models pass training parameters
[06:23 AM]: Training Augustus using PASA gene models
[06:23 AM]: Augustus initial training results:
  Feature       Specificity   Sensitivity
  nucleotides   97.0%         85.4%
  exons         49.1%         40.3%
  genes         47.0%         38.2%
[06:23 AM]: Accuracy seems low, you can try to improve by passing the --optimize_augustus option.
[06:23 AM]: Running Augustus gene prediction using awesome_rna parameters
[06:25 AM]: 1,609 predictions from Augustus
[06:25 AM]: Pulling out high quality Augustus predictions
[06:25 AM]: Found 31 high quality predictions from Augustus (>90% exon evidence)
[06:25 AM]: Running CodingQuarry prediction using stringtie alignments
[06:34 AM]: 1,655 predictions from CodingQuarry
[06:34 AM]: Running SNAP gene prediction, using training data: rna-seq/predict_misc/final_training_models.gff3
[06:35 AM]: 0 predictions from SNAP
[06:35 AM]: SNAP prediction failed, moving on without result
[06:35 AM]: Running GlimmerHMM gene prediction, using training data: rna-seq/predict_misc/final_training_models.gff3
[06:36 AM]: 1,813 predictions from GlimmerHMM
[06:36 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        1578
  Augustus HiQ   2        31
  CodingQuarry   2        1655
  GeneMark       1        1533
  GlimmerHMM     1        1813
  pasa           6        580
  Total          -        7190
[06:36 AM]: Running EVM commands with 3 CPUs
[06:42 AM]: Converting to GFF3 and collecting all EVM results
[06:42 AM]: 1,806 total gene models from EVM
[06:42 AM]: Generating protein fasta files from 1,806 EVM models
[06:42 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[06:42 AM]: Found 168 gene models to remove: 0 too short; 0 span gaps; 248 transposable elements
[06:42 AM]: 1,638 gene models remaining
[06:42 AM]: Predicting tRNAs
[06:42 AM]: 111 tRNAscan models are valid (non-overlapping)
[06:42 AM]: Generating GenBank tbl annotation file
[06:42 AM]: Converting to final Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[06:43 AM]: Collecting final annotation files for 1,749 total gene models
[06:43 AM]: Funannotate predict is finished, output files are in the rna-seq/predict_results folder
[06:43 AM]: Your next step to capture UTRs and update annotation using PASA:

  funannotate update -i rna-seq --cpus 4

[06:43 AM]: Training parameters file saved: rna-seq/predict_results/awesome_rna.parameters.json
[06:43 AM]: Add species parameters to database:

  funannotate species -s awesome_rna -a rna-seq/predict_results/awesome_rna.parameters.json

#########################################################
Now running `funannotate update` to run PASA-mediated UTR addition and multiple transcripts
CMD: funannotate update -i rna-seq --cpus 4
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[06:43 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[06:43 AM]: Running 1.7.2
[06:43 AM]: No NCBI SBT file given, will use default, for NCBI submissions pass one here '--sbt'
[06:43 AM]: Found relevant files in rna-seq/training, will re-use them:
    Single reads: rna-seq/training/single.fq.gz
    Single Q-trimmed reads: rna-seq/training/trimmomatic/trimmed_single.fastq.gz
    Single normalized reads: rna-seq/training/normalize/single.norm.fq
    Trinity results: rna-seq/training/funannotate_train.trinity-GG.fasta
    Long-read results: rna-seq/training/funannotate_long-reads.fasta
    PASA config file: rna-seq/training/pasa/alignAssembly.txt
    BAM alignments: rna-seq/training/funannotate_train.coordSorted.bam
    StringTie GTF: rna-seq/training/funannotate_train.stringtie.gtf
[06:43 AM]: Reannotating Awesome rna, NCBI accession: None
[06:43 AM]: Previous annotation consists of: 1,638 protein coding gene models and 111 non-coding gene models
[06:43 AM]: Aligning long reads to genome with minimap2
[06:43 AM]: Finding long-reads not represented in Trinity assemblies
[06:44 AM]: Adding 0 unique long-reads to Trinity assemblies
[06:44 AM]: Merging BAM files: rna-seq/update_misc/nano_mRNA.coordSorted.bam, rna-seq/update_misc/trinity.alignments.bam
[06:44 AM]: Converting transcript alignments to GFF3 format
[06:45 AM]: Converting Trinity transcript alignments to GFF3 format
[06:45 AM]: PASA database is SQLite: /local/ifs3_scratch/METAGENOMICS/jespinoz/Testing/test-rna_seq_88701/rna-seq/training/pasa/Awesome_rna
[06:45 AM]: Running PASA annotation comparison step 1
[06:49 AM]: Running PASA annotation comparison step 2
[06:53 AM]: Using Kallisto TPM data to determine which PASA gene models to select at each locus
[06:53 AM]: Building Kallisto index
[06:53 AM]: Mapping reads using pseudoalignment in Kallisto
[06:53 AM]: Parsing Kallisto results. Keeping alt-splicing transcripts if expressed at least 10.0% of highest transcript per locus.
[06:53 AM]: Wrote 1,646 transcripts derived from 1,643 protein coding loci.
[06:53 AM]: Validating gene models (renaming, checking translations, filtering, etc)
[06:53 AM]: Writing 1,751 loci to TBL format: dropped 0 overlapping, 0 too short, and 0 frameshift gene models
[06:53 AM]: Converting to Genbank format
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[06:53 AM]: Collecting final annotation files
[06:53 AM]: Parsing GenBank files...comparing annotation
[06:53 AM]: Updated annotation complete:
-------------------------------------------------------
Total Gene Models:  1,751
Total transcripts:  1,755
New Gene Models:    4
No Change:      1,528
Update UTRs:        219
Exons Changed:      0
Exons/CDS Changed:  0
Dropped Models:     0
CDS AED:        0.001
mRNA AED:       0.011
-------------------------------------------------------
[06:53 AM]: Funannotate update is finished, output files are in the rna-seq/update_results folder
[06:53 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (Docker required):
funannotate iprscan -i rna-seq -m docker -c 4

Run antiSMASH:
funannotate remote -i rna-seq -m antismash -e youremail@server.edu

Annotate Genome:
funannotate annotate -i rna-seq --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

#########################################################
SUCCESS: funannotate RNA-seq training/prediction test complete.
#########################################################

#########################################################

#########################################################
Running `funannotate annotate` unit testing
Downloading: https://osf.io/97pyn/download?version=1 Bytes: 341476
CMD: funannotate annotate --genbank Genome_one.gbk -o annotate --cpus 4 --iprscan genome_one.iprscan.xml --eggnog genome_one.emapper.annotations
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[06:54 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[06:54 AM]: Running 1.7.2
[06:54 AM]: No NCBI SBT file given, will use default, however if you plan to submit to NCBI, create one and pass it here '--sbt'
[06:54 AM]: Checking GenBank file for annotation
[06:54 AM]: Adding Functional Annotation to Genome one, NCBI accession: None
[06:54 AM]: Annotation consists of: 125 gene models
[06:54 AM]: 124 protein records loaded
[06:54 AM]: Running HMMer search of PFAM version 31.0
[06:54 AM]: 90 annotations added
[06:54 AM]: Running Diamond blastp search of UniProt DB version 2018_03
[06:58 AM]: 11 valid gene/product annotations from 13 total
[06:58 AM]: Existing Eggnog-mapper results found: annotate/annotate_misc/eggnog.emapper.annotations
[06:58 AM]: Parsing EggNog Annotations
[06:58 AM]: 132 COG and EggNog annotations added
[06:58 AM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.5
[06:58 AM]: 20 gene name and product description annotations added
[06:58 AM]: Running Diamond blastp search of MEROPS version 12.0
[06:58 AM]: 0 annotations added
[06:58 AM]: Annotating CAZYmes using HMMer search of dbCAN version 6.0
[06:59 AM]: 2 annotations added
[06:59 AM]: Annotating proteins with BUSCO dikarya models
[06:59 AM]: 6 annotations added
[06:59 AM]: Skipping phobius predictions, try funannotate remote -m phobius
[06:59 AM]: Predicting secreted proteins with SignalP
[06:59 AM]: 9 secretome and 0 transmembane annotations added
[06:59 AM]: Parsing InterProScan5 XML file
[06:59 AM]: Found 0 duplicated annotations, adding 635 valid annotations
[06:59 AM]: Converting to final Genbank format, good luck!
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
[06:59 AM]: Creating AGP file and corresponding contigs file
[06:59 AM]: Writing genome annotation table.
[06:59 AM]: Funannotate annotate has completed successfully!
-------------------------------------------------------
#########################################################
SUCCESS: `funannotate annotate` test complete.
#########################################################

#########################################################
Running `funannotate compare` unit testing
Downloading: https://osf.io/7s9xh/download?version=1 Bytes: 1020999
CMD: funannotate compare -i Genome_one.gbk Genome_two.gbk Genome_three.gbk -o compare --cpus 4 --run_dnds estimate --outgroup botrytis_cinerea.dikarya
#########################################################
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/funannotate2_env/lib/python2.7/site-packages/pkg_resources/py2_warn.py:19: UserWarning: ************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
-------------------------------------------------------
[06:59 AM]: OS: linux2, 2 cores, ~ 8 GB RAM. Python: 2.7.15
[06:59 AM]: Running 1.7.2
[07:00 AM]: Now parsing 3 genomes
[07:00 AM]: working on Genome one
[07:00 AM]: working on Genome two
[07:00 AM]: working on Genome three
[07:00 AM]: No secondary metabolite annotations found
[07:00 AM]: Summarizing PFAM domain results
[07:00 AM]: Summarizing InterProScan results
[07:00 AM]: Loading InterPro descriptions
[07:00 AM]: Summarizing MEROPS protease results
[07:00 AM]: found 4 MEROPS familes
[07:00 AM]: Summarizing CAZyme results
[07:00 AM]: found 5 CAZy familes
[07:00 AM]: Summarizing COG results
[07:00 AM]: Summarizing secreted protein results
[07:00 AM]: Summarizing fungal transcription factors
[07:00 AM]: Running GO enrichment for each genome
[07:04 AM]: Running orthologous clustering tool, ProteinOrtho.  This may take awhile...
[07:04 AM]: Calculating dN/dS ratios for each ortholog group, 128 orthologous groups
[07:05 AM]: Compiling all annotations for each genome
[07:05 AM]: Inferring phylogeny using RAxML
[07:05 AM]: Found 1 single copy BUSCO orthologs, will use all to infer phylogeny
[07:24 AM]: Compressing results to output file: compare.tar.gz
[07:25 AM]: Funannotate compare completed successfully!
#########################################################
SUCCESS: `funannotate compare` test complete.
#########################################################
nextgenusfs commented 4 years ago

Looks like something is wrong with your snap install -- I'm not sure what though.