Closed ivagljiva closed 11 months ago
I noticed we don't have a CITATION output for these programs. I think it would be nice to add one to anvi-run-ncbi-cogs
, though of course the citation is version-dependent.
I will add one, using the following references for each version:
Hey @ivagljiva, thank you very much for adding support for this. I think it would be nice to add a few lines of text into the anvi-setup-ncbi-cogs and anvi-run-ncbi-cogs to make sure there is some information somewhere about which databases are downloaded.
By the way, anvio/data/misc/checksum.md5.txt
is a very vague filename and I feel like we will forget about it in the long run. Shall we rename it to anvio/data/misc/CHECKSUMS-FOR-COG-DATA.txt
and change affected code?
Thank you again!
(I promise I will get to the citations issue at some point to offer a global solution for it :))
Citation lines added :)
By the way, anvio/data/misc/checksum.md5.txt is a very vague filename and I feel like we will forget about it in the long run. Shall we rename it to anvio/data/misc/CHECKSUMS-FOR-COG-DATA.txt and change affected code?
This sounds like a good idea. I will do it.
Side note: there are also a few files in there that we don't seem to use in the COGs code, namely:
genomes2003-2014.tab
prot2003-2014.gi2gbk.tab
prot2003-2014.tab
from the file names, they look like they belong to the COG14 release. I will leave them alone for now, but if you think we should remove them from anvio/data/misc/checksum.md5.txt
, let me know :)
I tested all version options, and they work with the new checksum file name (which only applies to COG14 and arCOG14, but I tested COG20 anyway). I will merge it now :)
🚀 THANK YOU 🚀 :)
This PR adds a new version of COG data,
arCOG14
, toanvi-setup-ncbi-cogs
andanvi-run-ncbi-cogs
. This database is the 'Archaeal Clusters of Orthologous Groups', last released in 2014 and described in the paper Makarova, Wolf, and Koonnin 2015.The only changes that were required to add this were: 1) adding the relevant files from https://ftp.ncbi.nih.gov/pub/wolf/COGs/arCOG/ to anvi'o's list of known COG versions, and 2) adding a few clauses for version-specific processing of these files in either program. The data formatting in this database seems to be a strange hybrid between COG14 and COG20, so in many cases, the previous clauses for one of those versions worked just fine.
There was one hiccup: a portion of the
ar14.arCOG.csv
file was incomplete. @meren and I confirmed that all lines starting from 388147 had a much shorter number of fields and were missing the critical arCOG ID field which we use to match protein sequences to their COG IDs. In order to bypass this issue, I wrote a very stupid hack for skipping the affected lines. Luckily, the downstream code is very smart and handles the missing protein IDs. So all that happens is that users see warnings like this when they set uparCOG14
withanvi-setup-ncbi-cogs
:And later when they annotate with
arCOG14
usinganvi-run-ncbi-cogs
, they see a warning like this:But otherwise everything is fine.
You can test the new code by running the following (multithreading with
-T
recommended):