nextgenusfs
commented
4 years ago Did you rebuild the databases with funannotate setup -i repeats —force? Otherwise try to use diamond v0.9.24. Either the database was created with different version of diamond or there the version you are running has an XML format bug. Other than that it also seems like busco is not finding enough models, I’m away from computer for next few days so I can’t check what it should be finding, but another test is to run funannotate test -t busco.
hyphaltip
gengce
Are you using the latest release? If you are not using the latest release of funannotate, please upgrade, if bug persists then report here.
Describe the bug funannotate predict still problem
[03:46 PM]: Summary of gene models passed to EVM (weights): Source Weight Count Augustus 1 1132 Augustus HiQ 2 277
main()
File "/opt/biosoft/miniconda/envs/funannotate/bin/funannotate", line 647, in main
mod.main(arguments)
File "/opt/biosoft/miniconda/envs/funannotate/lib/python2.7/site-packages/funannotate/predict.py", line 1755, in main
os.path.join(args.out, 'predict_misc'), Blast_rep_remove)
File "/opt/biosoft/miniconda/envs/funannotate/lib/python2.7/site-packages/funannotate/library.py", line 4429, in RepeatBlast
with open(blast_tmp, 'r') as results:
IOError: [Errno 2] No such file or directory: u'annotate/predict_misc/repeats.xml'
GeneMark 1 1588 GlimmerHMM 1 11
snap 1 3
Total - 3011 [03:46 PM]: Running EVM commands with 44 CPUs [03:47 PM]: Converting to GFF3 and collecting all EVM results [03:47 PM]: 1,212 total gene models from EVM [03:47 PM]: Generating protein fasta files from 1,212 EVM models [03:47 PM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc). Traceback (most recent call last): File "/opt/biosoft/miniconda/envs/funannotate/bin/funannotate", line 657, in
What command did you issue? funannotate test -t predict
Logfiles Please provide relavent log files of the error. [11/29/19 15:40:06]: /opt/biosoft/miniconda/envs/funannotate/bin/funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 80 --species Awesome testicus
[11/29/19 15:40:06]: OS: linux2, 80 cores, ~ 529 GB RAM. Python: 2.7.15 [11/29/19 15:40:06]: Running funannotate v1.7.1 [11/29/19 15:40:06]: {'genemark': 1, 'hiq': 2, 'glimmerhmm': 1, 'pasa': 6, 'snap': 1, 'transcripts': 1, 'proteins': 1, 'codingquarry': 2, 'augustus': 1} [11/29/19 15:40:07]: {'genemark': 'selftraining', 'snap': 'busco', 'glimmerhmm': 'busco', 'augustus': 'pretrained'} [11/29/19 15:40:07]: Parsed training data, run ab-initio gene predictors as follows: [11/29/19 15:40:07]: CodingQuarry will be skipped --> --rna_bam required for training [11/29/19 15:40:09]: {'genemark': 1, 'hiq': 2, 'glimmerhmm': 1, 'pasa': 6, 'snap': 1, 'transcripts': 1, 'proteins': 1, 'codingquarry': 0, 'augustus': 1} [11/29/19 15:40:09]: Loading genome assembly and parsing soft-masked repetitive sequences [11/29/19 15:40:09]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
11/29/19 15:40:31: Running GeneMark-ES on assembly 11/29/19 15:40:31: /opt/biosoft/gm_et_linux_64/gmes_petap/gmes_petap.pl --ES --max_intron 3000 --soft_mask 2000 --cores 64 --sequence /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa --fungus
11/29/19 15:43:11: perl /opt/biosoft/miniconda/envs/funannotate/opt/evidencemodeler-1.1.1/EvmUtils/misc/augustus_GFF3_to_EVM_GFF3.pl annotate/predict_misc/genemark.gff 11/29/19 15:43:11: 1,588 predictions from GeneMark 11/29/19 15:43:11: Running BUSCO to find conserved gene models for training ab-initio predictors [11/29/19 15:43:51]: 13 valid BUSCO predictions found, now formatting for EVM [11/29/19 15:43:52]: bedtools intersect -a annotate/predict_misc/genemark.evm.gff3 -b annotate/predict_misc/buscos.bed [11/29/19 15:43:52]: bedtools intersect -a /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/protein_alignments.gff3 -b annotate/predict_misc/buscos.bed [11/29/19 15:43:57]: 13 total gene models from EVM, now validating with BUSCO HMM search [11/29/19 15:43:57]: /opt/biosoft/miniconda/envs/funannotate/opt/evidencemodeler-1.1.1/EvmUtils/gff3_file_to_proteins.pl /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/busco.evm.gff3 /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa [11/29/19 15:44:29]: 13 BUSCO predictions validated [11/29/19 15:44:29]: Running Augustus gene prediction using saccharomyces parameters [11/29/19 15:45:47]: perl /opt/biosoft/miniconda/envs/funannotate/opt/evidencemodeler-1.1.1/EvmUtils/misc/augustus_GFF3_to_EVM_GFF3.pl annotate/predict_misc/augustus.gff3 [11/29/19 15:45:47]: Pulling out high quality Augustus predictions [11/29/19 15:45:47]: Found 277 high quality predictions from Augustus (>90% exon evidence) [11/29/19 15:45:47]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3 [11/29/19 15:45:47]: 13 gene models to train snap on 5 scaffolds [11/29/19 15:45:47]: fathom /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/snap.training.zff /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/snap-training.scaffolds.fasta -categorize 1000 -min-intron 10 -max-intron 3000 [11/29/19 15:45:48]: fathom uni.ann uni.dna -export 1000 -plus [11/29/19 15:45:48]: forge export.ann export.dna [11/29/19 15:45:48]: perl /opt/biosoft/miniconda/envs/funannotate/bin/hmm-assembler.pl snap-trained annotate/predict_misc/snaptrain [11/29/19 15:45:48]: snap annotate/predict_misc/snap-trained.hmm /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa [11/29/19 15:46:03]: 3 predictions from SNAP [11/29/19 15:46:03]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3 [11/29/19 15:46:04]: trainGlimmerHMM /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/glimmer.exons -d annotate/predict_misc/glimmerhmm [11/29/19 15:46:06]: perl /opt/biosoft/miniconda/envs/funannotate/bin/glimmhmm.pl /opt/biosoft/miniconda/envs/funannotate/bin/glimmerhmm /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/glimmerhmm -g [11/29/19 15:46:24]: 11 predictions from GlimmerHMM [11/29/19 15:46:24]: Prediction sources: ['Augustus', 'HiQ', 'GeneMark', 'GlimmerHMM', 'snap'] [11/29/19 15:46:24]: Summary of gene models: {u'GeneMark': 1588, u'HiQ': 277, u'Augustus': 1132, u'GlimmerHMM': 11, u'snap': 3, u'total': 3011} [11/29/19 15:46:24]: EVM Weights: {'GeneMark': 1, 'HiQ': 2, 'proteins': 1, 'Augustus': 1, 'GlimmerHMM': 1, 'snap': 1} [11/29/19 15:46:24]: Summary of gene models passed to EVM (weights): [11/29/19 15:46:24]: Launching EVM via funannotate-runEVM.py [11/29/19 15:46:24]: /opt/biosoft/miniconda/envs/funannotate/bin/python /opt/biosoft/miniconda/envs/funannotate/lib/python2.7/site-packages/funannotate/aux_scripts/funannotate-runEVM.py annotate/logfiles/funannotate-EVM.log 80 /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/EVM --genome /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/genome.softmasked.fa --gene_predictions /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/gene_predictions.gff3 --weights /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/weights.evm.txt --protein_alignments /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/protein_alignments.gff3 --min_intron_length 10 /home/train/DATA/ProVinka/Sample_R19050872-RX1115-QD-zj1-MEFC342/test-predict_33184/annotate/predict_misc/evm.round1.gff3 [11/29/19 15:47:20]: 1,212 total gene models from EVM [11/29/19 15:47:20]: Generating protein fasta files from 1,212 EVM models [11/29/19 15:47:22]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc). [11/29/19 15:47:22]: diamond blastp --sensitive --query annotate/predict_misc/evm.round1.proteins.fa --threads 80 --out annotate/predict_misc/repeats.xml --db /home/train/DATA/FunannotateDB/repeats.dmnd --evalue 1e-10 --max-target-seqs 1 --outfmt 5
OS/Install Information (funannotate) [train@localhost Sample_R19050872-RX1115-QD-zj1-MEFC342]$ funannotate check --show-versions
Checking dependencies for 1.7.1
You are running Python v 2.7.15. Now checking python packages... biopython: 1.74 goatools: 0.9.9 matplotlib: 2.2.4 natsort: 6.2.0 numpy: 1.16.5 pandas: 0.24.2 psutil: 5.6.5 requests: 2.22.0 scikit-learn: 0.20.3 scipy: 1.2.1 seaborn: 0.9.0 All 11 python packages installed
You are running Perl v 5.026002. Now checking perl modules... Bio::Perl: 1.007002 Carp: 1.38 Clone: 0.42 DBD::SQLite: 1.64 DBD::mysql: 4.046 DBI: 1.642 DB_File: 1.852 Data::Dumper: 2.173 File::Basename: 2.85 File::Which: 1.23 Getopt::Long: 2.5 Hash::Merge: 0.300 JSON: 4.02 LWP::UserAgent: 6.39 Logger::Simple: 2.0 POSIX: 1.76 Parallel::ForkManager: 2.02 Pod::Usage: 1.69 Scalar::Util::Numeric: 0.40 Storable: 3.15 Text::Soundex: 3.05 Thread::Queue: 3.12 Tie::File: 1.02 URI::Escape: 3.31 YAML: 1.29 threads: 2.15 threads::shared: 1.56 All 27 Perl modules installed
Checking Environmental Variables... $FUNANNOTATE_DB=/home/train/DATA/FunannotateDB $PASAHOME=/opt/biosoft/miniconda/envs/funannotate/opt/pasa-2.4.1 $TRINITYHOME=/opt/biosoft/Trinity-v2.5.1 $EVM_HOME=/opt/biosoft/miniconda/envs/funannotate/opt/evidencemodeler-1.1.1 $AUGUSTUS_CONFIG_PATH=/opt/biosoft/miniconda/envs/funannotate/config/ $GENEMARK_PATH=/opt/biosoft/gm_et_linux_64/gmes_petap All 6 environmental variables are set
Checking external dependencies... PASA: 2.4.1 CodingQuarry: 2.0 Trinity: 2.8.5 augustus: 3.3.3 bamtools: bamtools 2.5.1 bedtools: bedtools v2.29.0 blat: BLAT v36 diamond: 0.9.14 emapper.py: 1.0.3-40-g41a8498 ete3: 3.1.1 exonerate: exonerate 2.4.0 fasta: no way to determine glimmerhmm: 3.0.4 gmap: 2017-11-15 gmes_petap.pl: 4.48_3.60_lic hisat2: 2.1.0 hmmscan: HMMER 3.2.1 (June 2018) hmmsearch: HMMER 3.2.1 (June 2018) java: 11.0.1 kallisto: 0.46.0 mafft: v7.407 (2018/Jul/23) makeblastdb: makeblastdb 2.2.31+ minimap2: 2.17-r941 proteinortho: 6.0.10 pslCDnaFilter: no way to determine salmon: salmon 0.15.0 samtools: samtools 1.9 signalp: 4.1 snap: 2006-07-28 stringtie: 2.0 tRNAscan-SE: 2.0.3 (April 2019) tantan: tantan 13 tbl2asn: no way to determine, likely 25.X tblastn: tblastn 2.2.31+ trimal: trimAl v1.4.rev15 build[2013-12-17] trimmomatic: 0.39 All 36 external dependencies are installed