Closed fmobegi closed 3 years ago
Just noticed that annotate.py
script uses v1 eggnog by defauls.. changed line 222 to IDi, DBi, OGi, Genei, COGi, Desci = getEggNogHeadersv2(input)
Yes I never finished the v2 parser because the format was changing and they still haven’t tagged a stable v2 release as far as I know.
I see, it causes problems down the line
[Mar 09 12:02 PM]: OS: Ubuntu 20.10, 12 cores, ~ 33 GB RAM. Python: 3.8.6
[Mar 09 12:02 PM]: Running 1.8.4
[Mar 09 12:02 PM]: Found existing output directory funannotate_output/15CUR002. Warning, will re-use any intermediate files found.
[Mar 09 12:02 PM]: Parsing input files
[Mar 09 12:02 PM]: Existing tbl found: funannotate_output/15CUR002/predict_results/Ascochyta_rabiei_15CUR002.tbl
[Mar 09 12:03 PM]: Adding Functional Annotation to Ascochyta rabiei, NCBI accession: None
[Mar 09 12:03 PM]: Annotation consists of: 9,878 gene models
[Mar 09 12:03 PM]: 9,747 protein records loaded
[Mar 09 12:03 PM]: Existing Pfam-A results found: funannotate_output/15CUR002/annotate_misc/annotations.pfam.txt
[Mar 09 12:03 PM]: 11,115 annotations added
[Mar 09 12:03 PM]: Running Diamond blastp search of UniProt DB version 2021_01
[Mar 09 12:03 PM]: 678 valid gene/product annotations from 984 total
[Mar 09 12:03 PM]: Existing Eggnog-mapper results found: funannotate_output/15CUR002/annotate_misc/eggnog.emapper.annotations
[Mar 09 12:03 PM]: Parsing EggNog Annotations
[Mar 09 12:03 PM]: 16,902 COG and EggNog annotations added
[Mar 09 12:03 PM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.66
[Mar 09 12:03 PM]: 678 gene name and product description annotations added
[Mar 09 12:03 PM]: Existing MEROPS results found: funannotate_output/15CUR002/annotate_misc/annotations.merops.txt
[Mar 09 12:03 PM]: 331 annotations added
[Mar 09 12:03 PM]: Existing CAZYme results found: funannotate_output/15CUR002/annotate_misc/annotations.dbCAN.txt
[Mar 09 12:03 PM]: 476 annotations added
[Mar 09 12:03 PM]: Existing BUSCO2 results found: funannotate_output/15CUR002/annotate_misc/annotations.busco.txt
[Mar 09 12:03 PM]: 1,243 annotations added
[Mar 09 12:03 PM]: Existing Phobius results found: funannotate_output/15CUR002/annotate_misc/phobius.results.txt
[Mar 09 12:03 PM]: Existing SignalP results found: funannotate_output/15CUR002/annotate_misc/signalp.results.txt
[Mar 09 12:03 PM]: 1,007 secretome and 2,186 transmembane annotations added
[Mar 09 12:03 PM]: Parsing InterProScan5 XML file
[Mar 09 12:03 PM]: Now parsing antiSMASH v6 results, finding SM clusters
[Mar 09 12:03 PM]: Found 28 clusters, 68 biosynthetic enyzmes, and 84 smCOGs predicted by antiSMASH
[Mar 09 12:03 PM]: Found 2,165 duplicated annotations, adding 61,726 valid annotations
[Mar 09 12:03 PM]: Converting to final Genbank format, good luck!
[Mar 09 12:04 PM]: Creating AGP file and corresponding contigs file
[Mar 09 12:04 PM]: Cross referencing SM cluster hits with MIBiG database version 1.4
[Mar 09 12:05 PM]: Creating tab-delimited SM cluster output
[Mar 09 12:05 PM]: Writing genome annotation table.
Traceback (most recent call last):
File "/home/fredrick/.local/bin/funannotate", line 8, in <module>
sys.exit(main())
File "/home/fredrick/.local/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
mod.main(arguments)
File "/home/fredrick/.local/lib/python3.8/site-packages/funannotate/annotate.py", line 1495, in main
lib.annotationtable(final_gbk, FUNDB, NoteHeaders, INTERPRO,
File "/home/fredrick/.local/lib/python3.8/site-packages/funannotate/library.py", line 7543, in annotationtable
desc = x + ':'+ resources.COGS.get(x)
TypeError: can only concatenate str (not "NoneType") to str
Quick fix.. desc = x + ':'+ str(resources.COGS.get(x))
for the line 7543. Works for now and the results look reasonable.
Looks like eggnog devs finally tagged a v2.x release -- I'll get it installed locally and take a look at output files and then should be able to update the parser to work with either v1 or v2 eggnog (hopefully).
I believe these are the two files generated with running emapper-2.0.8-2 in funannotate.
eggnog.emapper.annotations.seed_orthologs.txt eggnog.emapper.annotations.txt
Hi @fmobegi thanks for the reference files. I think I have it working I'm going to reopen issue as the modifications you made above might have made it so pipeline didn't crash but didn't fix the underlying differences between the v1 and v2 formats. I also added EC_number parsing from eggnog v2 data.
If you would be able to test that would be great, can install from master with pip. If all is working, I'll tag a new release in the next few days. Thanks.
Thanks @nextgenusfs . Everything works okay now.
Here is the progress from a few of the isolates I was analysing
-------------------------------------------------------
[Mar 12 01:03 PM]: OS: Ubuntu 20.10, 12 cores, ~ 33 GB RAM. Python: 3.8.6
[Mar 12 01:03 PM]: Running 1.8.4
[Mar 12 01:03 PM]: Found existing output directory funannotate_output/TR6417. Warning, will re-use any intermediate files found.
[Mar 12 01:03 PM]: Parsing input files
[Mar 12 01:03 PM]: Existing tbl found: funannotate_output/TR6417/predict_results/Ascochyta_rabiei_TR6417.tbl
[Mar 12 01:03 PM]: Adding Functional Annotation to Ascochyta rabiei, NCBI accession: None
[Mar 12 01:03 PM]: Annotation consists of: 9,766 gene models
[Mar 12 01:03 PM]: 9,633 protein records loaded
[Mar 12 01:03 PM]: Existing Pfam-A results found: funannotate_output/TR6417/annotate_misc/annotations.pfam.txt
[Mar 12 01:03 PM]: 10,843 annotations added
[Mar 12 01:03 PM]: Running Diamond blastp search of UniProt DB version 2021_01
[Mar 12 01:03 PM]: 683 valid gene/product annotations from 984 total
[Mar 12 01:03 PM]: Existing Eggnog-mapper results found: funannotate_output/TR6417/annotate_misc/eggnog.emapper.annotations
[Mar 12 01:03 PM]: Parsing EggNog Annotations
[Mar 12 01:03 PM]: 18,165 COG and EggNog annotations added
[Mar 12 01:03 PM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.66
[Mar 12 01:03 PM]: 2,451 gene name and product description annotations added
[Mar 12 01:03 PM]: Existing MEROPS results found: funannotate_output/TR6417/annotate_misc/annotations.merops.txt
[Mar 12 01:03 PM]: 331 annotations added
[Mar 12 01:03 PM]: Existing CAZYme results found: funannotate_output/TR6417/annotate_misc/annotations.dbCAN.txt
[Mar 12 01:03 PM]: 465 annotations added
[Mar 12 01:03 PM]: Existing BUSCO2 results found: funannotate_output/TR6417/annotate_misc/annotations.busco.txt
[Mar 12 01:03 PM]: 1,231 annotations added
[Mar 12 01:03 PM]: Existing Phobius results found: funannotate_output/TR6417/annotate_misc/phobius.results.txt
[Mar 12 01:03 PM]: Existing SignalP results found: funannotate_output/TR6417/annotate_misc/signalp.results.txt
[Mar 12 01:03 PM]: 973 secretome and 2,129 transmembane annotations added
[Mar 12 01:03 PM]: Parsing InterProScan5 XML file
[Mar 12 01:03 PM]: Now parsing antiSMASH v6 results, finding SM clusters
[Mar 12 01:03 PM]: Found 25 clusters, 60 biosynthetic enyzmes, and 73 smCOGs predicted by antiSMASH
[Mar 12 01:03 PM]: Found 0 duplicated annotations, adding 65,558 valid annotations
[Mar 12 01:03 PM]: Converting to final Genbank format, good luck!
[Mar 12 01:04 PM]: Creating AGP file and corresponding contigs file
[Mar 12 01:04 PM]: Cross referencing SM cluster hits with MIBiG database version 1.4
[Mar 12 01:04 PM]: Creating tab-delimited SM cluster output
[Mar 12 01:04 PM]: Writing genome annotation table.
[Mar 12 01:05 PM]: Funannotate annotate has completed successfully!
We need YOUR help to improve gene names/product descriptions:
0 gene/products names MUST be fixed, see funannotate_output/TR6417/annotate_results/Gene2Products.must-fix.txt
2 gene/product names need to be curated, see funannotate_output/TR6417/annotate_results/Gene2Products.need-curating.txt
100 gene/product names passed but are not in Database, see funannotate_output/TR6417/annotate_results/Gene2Products.new-names-passed.txt
Please consider contributing a PR at https://github.com/nextgenusfs/gene2product
-------------------------------------------------------
-------------------------------------------------------
[Mar 12 01:05 PM]: OS: Ubuntu 20.10, 12 cores, ~ 33 GB RAM. Python: 3.8.6
[Mar 12 01:05 PM]: Running 1.8.4
[Mar 12 01:05 PM]: Found existing output directory funannotate_output/TR9544. Warning, will re-use any intermediate files found.
[Mar 12 01:05 PM]: Parsing input files
[Mar 12 01:05 PM]: Existing tbl found: funannotate_output/TR9544/predict_results/Ascochyta_rabiei_TR9544.tbl
[Mar 12 01:05 PM]: Adding Functional Annotation to Ascochyta rabiei, NCBI accession: None
[Mar 12 01:05 PM]: Annotation consists of: 10,408 gene models
[Mar 12 01:05 PM]: 10,262 protein records loaded
[Mar 12 01:05 PM]: Existing Pfam-A results found: funannotate_output/TR9544/annotate_misc/annotations.pfam.txt
[Mar 12 01:05 PM]: 11,616 annotations added
[Mar 12 01:05 PM]: Running Diamond blastp search of UniProt DB version 2021_01
[Mar 12 01:05 PM]: 718 valid gene/product annotations from 1,047 total
[Mar 12 01:05 PM]: Existing Eggnog-mapper results found: funannotate_output/TR9544/annotate_misc/eggnog.emapper.annotations
[Mar 12 01:05 PM]: Parsing EggNog Annotations
[Mar 12 01:05 PM]: 19,382 COG and EggNog annotations added
[Mar 12 01:05 PM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.66
[Mar 12 01:05 PM]: 2,589 gene name and product description annotations added
[Mar 12 01:05 PM]: Existing MEROPS results found: funannotate_output/TR9544/annotate_misc/annotations.merops.txt
[Mar 12 01:05 PM]: 344 annotations added
[Mar 12 01:05 PM]: Existing CAZYme results found: funannotate_output/TR9544/annotate_misc/annotations.dbCAN.txt
[Mar 12 01:05 PM]: 496 annotations added
[Mar 12 01:05 PM]: Existing BUSCO2 results found: funannotate_output/TR9544/annotate_misc/annotations.busco.txt
[Mar 12 01:05 PM]: 1,265 annotations added
[Mar 12 01:05 PM]: Existing Phobius results found: funannotate_output/TR9544/annotate_misc/phobius.results.txt
[Mar 12 01:05 PM]: Existing SignalP results found: funannotate_output/TR9544/annotate_misc/signalp.results.txt
[Mar 12 01:05 PM]: 1,065 secretome and 2,300 transmembane annotations added
[Mar 12 01:05 PM]: Parsing InterProScan5 XML file
[Mar 12 01:05 PM]: Now parsing antiSMASH v6 results, finding SM clusters
[Mar 12 01:05 PM]: Found 30 clusters, 73 biosynthetic enyzmes, and 83 smCOGs predicted by antiSMASH
[Mar 12 01:05 PM]: Found 0 duplicated annotations, adding 69,929 valid annotations
[Mar 12 01:05 PM]: Converting to final Genbank format, good luck!
[Mar 12 01:06 PM]: Creating AGP file and corresponding contigs file
[Mar 12 01:06 PM]: Cross referencing SM cluster hits with MIBiG database version 1.4
[Mar 12 01:06 PM]: Creating tab-delimited SM cluster output
[Mar 12 01:06 PM]: Writing genome annotation table.
[Mar 12 01:06 PM]: Funannotate annotate has completed successfully!
We need YOUR help to improve gene names/product descriptions:
0 gene/products names MUST be fixed, see funannotate_output/TR9544/annotate_results/Gene2Products.must-fix.txt
3 gene/product names need to be curated, see funannotate_output/TR9544/annotate_results/Gene2Products.need-curating.txt
97 gene/product names passed but are not in Database, see funannotate_output/TR9544/annotate_results/Gene2Products.new-names-passed.txt
Please consider contributing a PR at https://github.com/nextgenusfs/gene2product
-------------------------------------------------------
-------------------------------------------------------
[Mar 12 01:06 PM]: OS: Ubuntu 20.10, 12 cores, ~ 33 GB RAM. Python: 3.8.6
[Mar 12 01:06 PM]: Running 1.8.4
[Mar 12 01:06 PM]: Found existing output directory funannotate_output/TR9571. Warning, will re-use any intermediate files found.
[Mar 12 01:06 PM]: Parsing input files
[Mar 12 01:06 PM]: Existing tbl found: funannotate_output/TR9571/predict_results/Ascochyta_rabiei_TR9571.tbl
[Mar 12 01:07 PM]: Adding Functional Annotation to Ascochyta rabiei, NCBI accession: None
[Mar 12 01:07 PM]: Annotation consists of: 11,226 gene models
[Mar 12 01:07 PM]: 10,946 protein records loaded
[Mar 12 01:07 PM]: Existing Pfam-A results found: funannotate_output/TR9571/annotate_misc/annotations.pfam.txt
[Mar 12 01:07 PM]: 12,652 annotations added
[Mar 12 01:07 PM]: Running Diamond blastp search of UniProt DB version 2021_01
[Mar 12 01:07 PM]: 774 valid gene/product annotations from 1,265 total
[Mar 12 01:07 PM]: Existing Eggnog-mapper results found: funannotate_output/TR9571/annotate_misc/eggnog.emapper.annotations
[Mar 12 01:07 PM]: Parsing EggNog Annotations
[Mar 12 01:07 PM]: 21,724 COG and EggNog annotations added
[Mar 12 01:07 PM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.66
[Mar 12 01:07 PM]: 2,778 gene name and product description annotations added
[Mar 12 01:07 PM]: Existing MEROPS results found: funannotate_output/TR9571/annotate_misc/annotations.merops.txt
[Mar 12 01:07 PM]: 386 annotations added
[Mar 12 01:07 PM]: Existing CAZYme results found: funannotate_output/TR9571/annotate_misc/annotations.dbCAN.txt
[Mar 12 01:07 PM]: 516 annotations added
[Mar 12 01:07 PM]: Existing BUSCO2 results found: funannotate_output/TR9571/annotate_misc/annotations.busco.txt
[Mar 12 01:07 PM]: 1,336 annotations added
[Mar 12 01:07 PM]: Existing Phobius results found: funannotate_output/TR9571/annotate_misc/phobius.results.txt
[Mar 12 01:07 PM]: Existing SignalP results found: funannotate_output/TR9571/annotate_misc/signalp.results.txt
[Mar 12 01:07 PM]: 1,087 secretome and 2,377 transmembane annotations added
[Mar 12 01:07 PM]: Parsing InterProScan5 XML file
[Mar 12 01:07 PM]: Now parsing antiSMASH v6 results, finding SM clusters
[Mar 12 01:07 PM]: Found 27 clusters, 67 biosynthetic enyzmes, and 83 smCOGs predicted by antiSMASH
[Mar 12 01:07 PM]: Found 11,678 duplicated annotations, adding 76,672 valid annotations
[Mar 12 01:07 PM]: Converting to final Genbank format, good luck!
[Mar 12 01:08 PM]: Creating AGP file and corresponding contigs file
[Mar 12 01:08 PM]: Cross referencing SM cluster hits with MIBiG database version 1.4
[Mar 12 01:08 PM]: Creating tab-delimited SM cluster output
[Mar 12 01:08 PM]: Writing genome annotation table.
[Mar 12 01:09 PM]: Funannotate annotate has completed successfully!
We need YOUR help to improve gene names/product descriptions:
0 gene/products names MUST be fixed, see funannotate_output/TR9571/annotate_results/Gene2Products.must-fix.txt
3 gene/product names need to be curated, see funannotate_output/TR9571/annotate_results/Gene2Products.need-curating.txt
103 gene/product names passed but are not in Database, see funannotate_output/TR9571/annotate_results/Gene2Products.new-names-passed.txt
Please consider contributing a PR at https://github.com/nextgenusfs/gene2product
With this update funannotate compare
also runs well without any hiccups.
Hi there,
I just run funannotate annotate
(v1.8.7) with EggNOG mapper results generated externally (emapper.py v2.1.2) and funannotate fails to parse every single annotation. Apparently some changes have been made to the output files of emapper in the last release (https://github.com/eggnogdb/eggnog-mapper/wiki/eggNOG-mapper-v2.1.2#v212)
I believe I'm seeing the same issue as @xvazquezc : eggnog.emapper.annotations
has 6307 lines, but annotations.eggnog.txt
has zero.
Using funannotate
v1.8.7, with built-in emapper
annotation.
I had a look at your previous fixing commit (https://github.com/nextgenusfs/funannotate/commit/bde15b399870cce096a15a98b699b7712e4a8726) hoping for a quick fix, but looks like this will need more than 5 minutes I had now 🙂
Can one of you send me the first like 10 lines of your emapper.annotations file so I can see the headers, names, etc.
## Fri May 14 17:34:25 2021
## emapper-e6ac7f2
## funannotate/bin/emapper.py -m diamond -i genome.proteins.fasta -o eggnog --cpu 8
##
#query seed_ortholog evalue score eggNOG_OGs max_annot_lvl COG_category Description Preferred_name GOs EC KEGG_ko KEGG_Pathway KEGG_Module KEGG_Reaction KEGG_rclass BRITE KEGG_TC CAZy BiGG_Reaction PFAMs
FUN_000001-T1 64363.EME43602 3.5e-244 685.0 KOG0254@1|root,KOG0254@2759|Eukaryota,39TDK@33154|Opisthokonta,3NWTD@4751|Fungi,3QM8J@4890|Ascomycota,1ZZI9@147541|Dothideomycetes,3MH72@451867|Dothideomycetidae 4751|Fungi U Major Facilitator Superfamily - GO:0003674,GO:0005215,GO:0005575,GO:0005623,GO:0005886,GO:0006810,GO:0006811,GO:0006812,GO:0008150,GO:0008324,GO:0008519,GO:0015075,GO:0015696,GO:0016020,GO:0016021,GO:0022857,GO:0031224,GO:0034220,GO:0044425,GO:0044464,GO:0051179,GO:0051234,GO:0055085,GO:0071705,GO:0071944,GO:0072488,GO:0098655 - - - - - - - - - - MFS_1,Pkinase
FUN_000002-T1 64363.EME43601 3.48e-279 812.0 COG1020@1|root,KOG1178@2759|Eukaryota,39REN@33154|Opisthokonta,3NTWI@4751|Fungi,3QJNI@4890|Ascomycota,1ZZPI@147541|Dothideomycetes,3MGA9@451867|Dothideomycetidae 4751|Fungi I AMP-binding enzyme - - - - - - - - - - - - AMP-binding,DIT1_PvcA,NAD_binding_4,PP-binding,Transferase
FUN_000003-T1 101852.XP_008088089.1 8.04e-125 388.0 COG0580@1|root,KOG0224@2759|Eukaryota,392UK@33154|Opisthokonta,3NZK7@4751|Fungi,3RJQE@4890|Ascomycota,210YQ@147548|Leotiomycetes 4751|Fungi G Major intrinsic protein - GO:0003674,GO:0005215,GO:0005372,GO:0005575,GO:0005623,GO:0006810,GO:0006833,GO:0008150,GO:0008643,GO:0015144,GO:0015166,GO:0015168,GO:0015250,GO:0015267,GO:0015318,GO:0015791,GO:0015793,GO:0015850,GO:0016020,GO:0016021,GO:0022803,GO:0022838,GO:0022857,GO:0031224,GO:0034219,GO:0042044,GO:0044425,GO:0044464,GO:0051179,GO:0051234,GO:0055085,GO:0071702,GO:0071944,GO:1901618 - ko:K03441 - - - - ko00000,ko02000 1.A.8 - - MIP,Mis12
FUN_000004-T1 101852.XP_008081284.1 9.97e-67 236.0 2AXX5@1|root,2S01X@2759|Eukaryota,393E9@33154|Opisthokonta,3NZWW@4751|Fungi,3QRKY@4890|Ascomycota,20Z21@147548|Leotiomycetes 4751|Fungi S DNA replication regulator SLD3 - GO:0000228,GO:0000278,GO:0000785,GO:0000790,GO:0005575,GO:0005622,GO:0005623,GO:0005634,GO:0005654,GO:0005694,GO:0006139,GO:0006259,GO:0006260,GO:0006261,GO:0006270,GO:0006725,GO:0006807,GO:0007049,GO:0008150,GO:0008152,GO:0009058,GO:0009059,GO:0009987,GO:0022402,GO:0031261,GO:0031974,GO:0031981,GO:0032991,GO:0032993,GO:0033260,GO:0034641,GO:0034645,GO:0043170,GO:0043226,GO:0043227,GO:0043228,GO:0043229,GO:0043231,GO:0043232,GO:0043233,GO:0044237,GO:0044238,GO:0044249,GO:0044260,GO:0044422,GO:0044424,GO:0044427,GO:0044428,GO:0044446,GO:0044451,GO:0044454,GO:0044464,GO:0044786,GO:0046483,GO:0070013,GO:0071704,GO:0090304,GO:1901360,GO:1901576,GO:1902292,GO:1902315,GO:1902969,GO:1902975,GO:1903047 - ko:K10731 - - - - ko00000,ko03032 - - - APH,SLD3
FUN_000006-T1 140110.NechaP101753 2.75e-169 482.0 KOG1339@1|root,KOG1339@2759|Eukaryota,39ZWM@33154|Opisthokonta,3NXD8@4751|Fungi,3QMYU@4890|Ascomycota,213QU@147550|Sordariomycetes,3TH3H@5125|Hypocreales,1FXHE@110618|Nectriaceae 4751|Fungi O Belongs to the peptidase A1 family - - 3.4.23.1,3.4.23.34 ko:K01382,ko:K06002 ko04142,ko04974,map04142,map04974 - - - ko00000,ko00001,ko01000,ko01002 - - - Asp
FUN_000007-T1 698440.XP_007293743.1 1.47e-24 110.0 2ETTM@1|root,2SW2Y@2759|Eukaryota,3A3JK@33154|Opisthokonta,3P8ZI@4751|Fungi,3QXHI@4890|Ascomycota,210KG@147548|Leotiomycetes 4751|Fungi - - - - - - - - - - - - - - -
FUN_000008-T1 101852.XP_008081286.1 3.68e-97 290.0 KOG1534@1|root,KOG1534@2759|Eukaryota,38BEG@33154|Opisthokonta,3NW4B@4751|Fungi,3QM4U@4890|Ascomycota,20WXW@147548|Leotiomycetes 4751|Fungi K Conserved hypothetical ATP binding protein - GO:0000070,GO:0000278,GO:0000280,GO:0000819,GO:0003674,GO:0003824,GO:0003924,GO:0005048,GO:0005488,GO:0006606,GO:0006810,GO:0006886,GO:0006913,GO:0006996,GO:0007049,GO:0007059,GO:0007062,GO:0007064,GO:0008104,GO:0008150,GO:0009987,GO:0015031,GO:0015833,GO:0016043,GO:0016462,GO:0016787,GO:0016817,GO:0016818,GO:0017038,GO:0017111,GO:0022402,GO:0033036,GO:0033218,GO:0033365,GO:0034504,GO:0034613,GO:0042277,GO:0042886,GO:0045184,GO:0046907,GO:0048285,GO:0051169,GO:0051170,GO:0051179,GO:0051234,GO:0051276,GO:0051641,GO:0051649,GO:0070727,GO:0071702,GO:0071705,GO:0071840,GO:0072594,GO:0098813,GO:0140014,GO:1903047 - ko:K06883 - - - - ko00000 - - - ATP_bind_1,FAA_hydrolase,SRPRB
FUN_000009-T1 655981.L8G9P6 7.09e-257 707.0 COG0192@1|root,KOG1506@2759|Eukaryota,38DWH@33154|Opisthokonta,3NUD5@4751|Fungi,3QM99@4890|Ascomycota,20WIM@147548|Leotiomycetes 4751|Fungi H Catalyzes the formation of S-adenosylmethionine from methionine and ATP SAM2 GO:0000096,GO:0000166,GO:0003674,GO:0003824,GO:0004478,GO:0005488,GO:0005524,GO:0005575,GO:0005622,GO:0005623,GO:0005737,GO:0005829,GO:0006082,GO:0006520,GO:0006555,GO:0006556,GO:0006732,GO:0006790,GO:0006807,GO:0008144,GO:0008150,GO:0008152,GO:0009058,GO:0009066,GO:0009108,GO:0009987,GO:0010494,GO:0016740,GO:0016765,GO:0017076,GO:0017144,GO:0019752,GO:0030554,GO:0032553,GO:0032555,GO:0032559,GO:0032991,GO:0035639,GO:0035770,GO:0036094,GO:0036464,GO:0043167,GO:0043168,GO:0043226,GO:0043228,GO:0043229,GO:0043232,GO:0043436,GO:0044237,GO:0044238,GO:0044249,GO:0044272,GO:0044281,GO:0044424,GO:0044444,GO:0044464,GO:0046500,GO:0051186,GO:0051188,GO:0071704,GO:0097159,GO:0097367,GO:1901265,GO:1901363,GO:1901564,GO:1901576,GO:1901605,GO:1990904 2.5.1.6 ko:K00789 ko00270,ko01100,ko01110,ko01230,map00270,map01100,map01110,map01230 M00034,M00035,M00368,M00609 R00177,R04771 RC00021,RC01211 ko00000,ko00001,ko00002,ko01000 - - iMM904.YLR180W,iND750.YLR180W S-AdoMet_synt_C,S-AdoMet_synt_M,S-AdoMet_synt_N
So is emapper-e6ac7f2
their latest in master? Perhaps that's actually the issue as I have the parser setup to look for a version number to parse.
https://github.com/nextgenusfs/funannotate/blob/master/funannotate/annotate.py#L280
And what does it say in the log file about what version it detected?
Okay, well this max_annot_lvl
is definitely a new header. Really sucks this keeps changing..... not sure I have the patience to constantly update this......
e6ac7f2
looks like a commit hash, but I don't see it on their master
branch.
within the conda env (installed with mamba
as recommended), pip list | grep eggnog
yields eggnog-mapper 2.1.2
.
emapper.py --version
(within the env) yields
$ emapper.py --version
emapper-e6ac7f2 / Expected eggNOG DB version: 5.0.2 / Installed eggNOG DB version: 5.0.2 / Local diamond version: diamond version 2.0.4 / Local MMseqs2 version: 113e3212c137d026e297c7540e1fcd039f6812b1
Here's a relevant log fragment:
[05/14/21 19:52:20]: Parsing EggNog Annotations
[05/14/21 19:52:20]: EggNog annotation detected as emapper ve6ac7f2 and DB prefix ENOG50
[05/14/21 19:52:20]: EggNog Parse ERROR: FUN_000001-T1 64363.EME43602 3.5e-244 685.0 KOG0254@1|root,KOG0254@2759|Eukaryota,39TDK@33154|Opisthokonta,3NWTD@4751|Fungi,3QM8J@4890|Ascomycota,1ZZI9@147541|Dothideomycetes,3MH72@451867|Dothideomycetidae 4751|Fungi U Major Facilitator Superfamily - GO:0003674,GO:0005215,GO:0005575,GO:0005623,GO:0005886,GO:0006810,GO:0006811,GO:0006812,GO:0008150,GO:0008324,GO:0008519,GO:0015075,GO:0015696,GO:0016020,GO:0016021,GO:0022857,GO:0031224,GO:0034220,GO:0044425,GO:0044464,GO:0051179,GO:0051234,GO:0055085,GO:0071705,GO:0071944,GO:0072488,GO:0098655 - - - - - - - - - - MFS_1,Pkinase
(and so on, it probably reports an error for each line)
oh, I was too slow with my logs, sorry 🙂
Okay, well that's not great for several reasons, but it actually got at least the v2 parser... problem is that hash is going to always evaluate as greater than... even if it was a hash from a version that is not there. But the problem here is the altered headers again. So I'll try to fix it for this one... but I'll open an issue -- I do not have the time to update this every few weeks.
>>> vers = 've6ac7f2'
>>> vers < ('2.0.0')
False
>>> vers > ('2.0.0')
True
eggnog-mapper
was installed as a funannotate
dependency with mamba
/conda
; as a quick issue work-around, I have tried installing eggnog-mapper 2.1.1
into this environment (instead of 2.1.2
), but there is no such version on bioconda anymore - I guess they replaced the older one.
Perhaps it's on pip?
Good thinking - it is!
Giving this a try, should work 🙂
$ emapper.py --version
emapper-2.1.1 / Expected eggNOG DB version: 5.0.2 / Installed eggNOG DB version: 5.0.2 / Local diamond version: diamond version 2.0.4 / Local MMseqs2 version: 113e3212c137d026e297c7540e1fcd039f6812b1
Yes, this workaround works.
Assuming funannotate
is installed into the same-named conda environment:
conda activate funannotate
pip install eggnog-mapper==2.1.1
In case it helps, in my case, I installed eggnog-mapper
with funannotate
in the same conda environment and both with conda. The emapper.py
version shows as 2.1.2 and the version is shown correct in the header of the .annotations
file, which should avoid the presence of the hash in the version number @spock has (?):
$ emapper.py --version
emapper-2.1.2
Just header and tail of the .annotations
file (+one result):
## Thu May 6 11:24:23 2021
## emapper-2.1.2
## /home/z3382651/miniconda3/envs/funannotate/bin/emapper.py --data_dir /share/bioinfo/z3382651/emapper_db/ --cpu 24 --scratch_dir /scratch/pbs.14720.clive.ramaciotti.unsw.edu.au/emapper --output emapper --dbmem -i /share/bioinfo/z3382651/ferrari/annotation/predict/predict_results/Penicillium_winsconsinense.proteins.fa
##
#query seed_ortholog evalue score eggNOG_OGs max_annot_lvl COG_category Description Preferred_name GOs EC KEGG_ko KEGG_Pathway KEGG_Module KEGG_Reaction KEGG_rclass BRITE KEGG_TC CAZy BiGG_Reaction PFAMs
PWIN_000002-T1 36630.CADNFIAP00003069 2.25e-238 662.0 COG1063@1|root,KOG0024@2759|Eukaryota,39UET@33154|Opisthokonta,3NUTE@4751|Fungi,3QQHE@4890|Ascomycota,20B2U@147545|Eurotiomycetes,3S2X3@5042|Eurotiales 4751|Fungi Q Alcohol dehydrogenase GroES-like domain - GO:0006950,GO:0008150,GO:0009636,GO:0009987,GO:0033554,GO:0042221,GO:0050896,GO:0051409,GO:0051410,GO:0051716,GO:0070458,GO:0070887,GO:0071500,GO:0097237,GO:0098754,GO:1901698,GO:1990748 - - - - - - - - - - ADH_N,ADH_N_assoc,ADH_zinc_N,Sacchrp_dh_NADP
.....
## 9382 queries scanned
## Total time (seconds): 200.2429313659668
## Rate: 46.85 q/s
This is the exact version and builds for both the emapper and funannotate
$ mamba list |grep "eggnog\|funannotate"
# packages in environment at /home/z3382651/miniconda3/envs/funannotate:
eggnog-mapper 2.1.2 pyhdfd78af_0 bioconda
funannotate 1.8.7 pyh5e36f6f_0 bioconda
Would somebody be able to test this latest commit for me on actual data? It should work for emapper v2.1.2 and greater. I don't have installed locally so would be great if somebody else can confirm this detects the right version and outputs the proper data. Thanks.
Am currently using emapper-2.1.4-2-6
with funannotate 1.8.7
and I don't have any problems.
(funannotate_env) fredrick@cicer:~$ funannotate check --show-versions
-------------------------------------------------------
Checking dependencies for 1.8.7
-------------------------------------------------------
You are running Python v 3.9.5. Now checking python packages...
biopython: 1.79
goatools: 1.1.6
matplotlib: 3.4.2
natsort: 7.1.1
numpy: 1.20.3
pandas: 1.2.4
psutil: 5.8.0
requests: 2.25.1
scikit-learn: 0.24.2
scipy: 1.6.3
seaborn: 0.11.1
All 11 python packages installed
You are running Perl v b'5.026002'. Now checking perl modules...
Bio::Perl: 1.007002
Carp: 1.38
Clone: 0.42
DBD::SQLite: 1.64
DBD::mysql: 4.046
DBI: 1.642
DB_File: 1.855
Data::Dumper: 2.173
File::Basename: 2.85
File::Which: 1.23
Getopt::Long: 2.5
Hash::Merge: 0.300
JSON: 4.02
LWP::UserAgent: 6.39
Logger::Simple: 2.0
POSIX: 1.76
Parallel::ForkManager: 2.02
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.15
Text::Soundex: 3.05
Thread::Queue: 3.12
Tie::File: 1.02
URI::Escape: 3.31
YAML: 1.29
threads: 2.15
threads::shared: 1.56
All 27 Perl modules installed
Checking Environmental Variables...
$FUNANNOTATE_DB=/ppgdata/fredrick/funannotate_db
$PASAHOME=/opt/miniconda3/envs/funannotate_env/opt/pasa-2.4.1
$TRINITY_HOME=/opt/miniconda3/envs/funannotate_env/opt/trinity-2.8.5
$EVM_HOME=/opt/miniconda3/envs/funannotate_env/opt/evidencemodeler-1.1.1
$AUGUSTUS_CONFIG_PATH=/opt/miniconda3/envs/funannotate_env/config/
$GENEMARK_PATH=/opt/gmes_linux_64
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
PASA: 2.4.1
CodingQuarry: 2.0
Trinity: 2.8.5
augustus: 3.3.3
bamtools: bamtools 2.5.1
bedtools: bedtools v2.30.0
blat: BLAT v36
diamond: 2.0.8
emapper.py: 2.1.4-2-6-g05f27b0
ete3: 3.1.2
exonerate: exonerate 2.4.0
fasta: no way to determine
glimmerhmm: 3.0.4
gmap: 2017-11-15
hisat2: 2.2.1
hmmscan: HMMER 3.3.2 (Nov 2020)
hmmsearch: HMMER 3.3.2 (Nov 2020)
java: 11.0.8-internal
kallisto: 0.46.1
mafft: v7.480 (2021/May/21)
makeblastdb: makeblastdb 2.2.31+
minimap2: 2.20-r1061
proteinortho: 6.0.31
pslCDnaFilter: no way to determine
salmon: salmon 0.14.1
samtools: samtools 1.10
signalp: 5.0b
snap: 2006-07-28
stringtie: 2.1.7
tRNAscan-SE: 2.0.7 (Oct 2020)
tantan: tantan 26
tbl2asn: no way to determine, likely 25.X
tblastn: tblastn 2.2.31+
trimal: trimAl v1.4.rev15 build[2013-12-17]
trimmomatic: 0.39
Hmm, well v1.8.7 may not be erring but I don't think it would be parsing the output properly, what sort of results are in annotate_misc/annotations.eggnog.txt
when you run it?
I've updates the script and everything works well.
(funannotate_env) fredrick@cicer:/ppgdata/fredrick/assembly_data/ascochyta/2021-06-01_Ascochyta_lentis_reannotation/A_lentis_Kewell$ funannotate annotate -i funannotate_out --cpus 10 --sbt template.sbt --iprscan Ascochyta_lentis_Kewell.interproscan.xml --antismash funannotate_out/fungi-984e0060-ebed-4ff9-8d73-97c7948355c2/Ascochyta_lentis_Kewell.gbk --phobius phobius.txt --signalp Ascochyta_lentis_Kewell.signalp5
-------------------------------------------------------
[Aug 02 09:36 AM]: OS: Ubuntu 20.04, 12 cores, ~ 33 GB RAM. Python: 3.8.10
[Aug 02 09:36 AM]: Running 1.8.8
[Aug 02 09:36 AM]: Found existing output directory funannotate_out. Warning, will re-use any intermediate files found.
[Aug 02 09:36 AM]: Parsing input files
[Aug 02 09:36 AM]: Existing tbl found: funannotate_out/update_results/Ascochyta_lentis_Kewell.tbl
[Aug 02 09:36 AM]: Adding Functional Annotation to Ascochyta lentis, NCBI accession: None
[Aug 02 09:36 AM]: Annotation consists of: 11,380 gene models
[Aug 02 09:36 AM]: 11,444 protein records loaded
[Aug 02 09:36 AM]: Existing Pfam-A results found: funannotate_out/annotate_misc/annotations.pfam.txt
[Aug 02 09:36 AM]: 12,891 annotations added
[Aug 02 09:36 AM]: Running Diamond blastp search of UniProt DB version 2021_02
[Aug 02 09:36 AM]: 756 valid gene/product annotations from 1,120 total
[Aug 02 09:36 AM]: Existing Eggnog-mapper results found: funannotate_out/annotate_misc/eggnog.emapper.annotations
[Aug 02 09:36 AM]: Parsing EggNog Annotations
[Aug 02 09:36 AM]: EggNog version parsed as 2.1.4-2-6-g05f27b0
[Aug 02 09:36 AM]: 21,649 COG and EggNog annotations added
[Aug 02 09:36 AM]: Combining UniProt/EggNog gene and product names using Gene2Product version 1.69
[Aug 02 09:36 AM]: 2,784 gene name and product description annotations added
[Aug 02 09:36 AM]: Existing MEROPS results found: funannotate_out/annotate_misc/annotations.merops.txt
[Aug 02 09:36 AM]: 384 annotations added
[Aug 02 09:36 AM]: Existing CAZYme results found: funannotate_out/annotate_misc/annotations.dbCAN.txt
[Aug 02 09:36 AM]: 555 annotations added
[Aug 02 09:36 AM]: Existing BUSCO2 results found: funannotate_out/annotate_misc/annotations.busco.txt
[Aug 02 09:36 AM]: 1,304 annotations added
[Aug 02 09:36 AM]: Existing Phobius results found: funannotate_out/annotate_misc/phobius.results.txt
[Aug 02 09:36 AM]: Existing SignalP results found: funannotate_out/annotate_misc/signalp.results.txt
[Aug 02 09:36 AM]: 1,225 secretome and 2,564 transmembane annotations added
[Aug 02 09:36 AM]: Parsing InterProScan5 XML file
[Aug 02 09:36 AM]: Now parsing antiSMASH v5 results, finding SM clusters
[Aug 02 09:36 AM]: Found 41 clusters, 116 biosynthetic enyzmes, and 137 smCOGs predicted by antiSMASH
[Aug 02 09:36 AM]: Found 0 duplicated annotations, adding 77,331 valid annotations
[Aug 02 09:36 AM]: Converting to final Genbank format, good luck!
[Aug 02 09:37 AM]: Creating AGP file and corresponding contigs file
[Aug 02 09:37 AM]: Cross referencing SM cluster hits with MIBiG database version 1.4
[Aug 02 09:38 AM]: Creating tab-delimited SM cluster output
[Aug 02 09:38 AM]: Writing genome annotation table.
[Aug 02 09:38 AM]: Funannotate annotate has completed successfully!
We need YOUR help to improve gene names/product descriptions:
0 gene/products names MUST be fixed, see funannotate_out/annotate_results/Gene2Products.must-fix.txt
1 gene/product names need to be curated, see funannotate_out/annotate_results/Gene2Products.need-curating.txt
108 gene/product names passed but are not in Database, see funannotate_out/annotate_results/Gene2Products.new-names-passed.txt
Please consider contributing a PR at https://github.com/nextgenusfs/gene2product
-------------------------------------------------------
(funannotate_env) fredrick@cicer:/ppgdata/fredrick/assembly_data/ascochyta/2021-06-01_Ascochyta_lentis_reannotation/A_lentis_Kewell$ head funannotate
funannotate-annotate.2789178.log funannotate-annotate.2813093.log funannotate-annotate.2814974.log funannotate_out/
(funannotate_env) fredrick@cicer:/ppgdata/fredrick/assembly_data/ascochyta/2021-06-01_Ascochyta_lentis_reannotation/A_lentis_Kewell$ head funannotate_out/
annotate_misc/ fungi-984e0060-ebed-4ff9-8d73-97c7948355c2/ predict_misc/ update_misc/
annotate_results/ logfiles/ predict_results/ update_results/
(funannotate_env) fredrick@cicer:/ppgdata/fredrick/assembly_data/ascochyta/2021-06-01_Ascochyta_lentis_reannotation/A_lentis_Kewell$ head funannotate_out/annotate_misc/annotations.eggnog.txt
AlKewell_000001-T1 note EggNog:ENOG502SB2X
AlKewell_000002-T1 EC_number 3.4.14.9
AlKewell_000002-T1 note EggNog:ENOG503Q3UA
AlKewell_000002-T1 note COG:O
AlKewell_000003-T1 EC_number 3.4.14.9
AlKewell_000003-T1 note EggNog:ENOG503Q3UA
AlKewell_000003-T1 note COG:O
AlKewell_000004-T1 note EggNog:ENOG503PCNJ
AlKewell_000004-T1 note COG:S
AlKewell_000005-T1 note EggNog:ENOG503NVNQ
Fantastic thanks @fmobegi for confirmation -- I'm going to tag a new release now.
Another thing you could check, python -m pip install git+https://github.com/nextgenusfs/funannotate.git
which is meant to install the latest code doesn't install under the latest python3.9.6 (the version tagged as 3.9).
ERROR: Package 'funannotate' requires a different Python: 3.9.6 not in '<3.9,>=3.6.0'
Had to altinstall python3.8 to get the pip install to work
python3.8 -m pip install git+https://github.com/nextgenusfs/funannotate.git
Thanks, yes I know it doesn't work in 3.9 yet -- haven't had time to figure out exactly why but its on my list. Its tagged properly in the setup.py which is why you see this error. Conda however doesn't not seem to respect these tags from pypi.
Am getting the following error when running
annotate
on my isolates. The same problem occurs even when providing pre-processed results using--antismash
and--eggnog
flags.