New downloads table for ~20 fully supported species with PAN-GO second download option

[suzialeksander]

As part of the doc drafted at https://docs.google.com/document/d/1j1zO-JHMaXi4ESrjTm_X1amLKdm3vO4szUaeB5LvxvA/edit#heading=h.25jngq5vebjo and the draft at https://github.com/geneontology/geneontology.github.io/pull/523, we need a new table for downloads.

comments below from @thomaspd on the draft doc:

The list of organisms is in the following table [or this can link to a separate page]:

• [columns should be organism name, taxonomy ID, identifier space, link to download full set, link to download IBAs]
• I suggest that links should be to GO website for ACTIVE ANNOTATION GROUPS that are the “authoritative source” and curate GO annotations directly for gene products (not complexes), protein binding annotations should (eventually, though not initially necessarily) be a separate file: human, mouse, rat, arabidopsis, zebrafish, X trop and X laevis, fly, worm, dicty, candida, E coli, pombe.
• I suggest that other links should point to UniProt/GOA downloads. I reviewed the experimental evidence in AmiGO and I’d propose including any organism with >350 manual (EXP+IBA) annotations
• So the table would have about 150 organisms (the 143 with IBAs, plus a few more that have >350 EXPs but no IBAs)
  

[suzialeksander]

@kltm @pgaudet it looks like the plan is to link GAFs and the paint GAFs. I can pull what we have out of Current and put it in a table, but that's only about 15 files. I don't know where to find the other IBA files. (?)

If these are all in the paint_other.gaf, we'll need a way to split that file- or @thomaspd was the plan to link the same giant file and have users split it themselves?

[thomaspd]

We should get a draft of the table ASAP, so we should fill in the table entries with the best files we have currently. So it's fine to have the link to the same giant file for now. When we have the file broken up by organism, we can update the table accordingly.

[suzialeksander]

To clarify, the links for the table:

• "all annotations": these are the organism/MOD files listed in http://current.geneontology.org/annotations, and if no specific file it goes to "dig your taxon out of filtered_goa_uniprot_all.gaf.gz" correct?

• "PAN-GO annotations only": the example in the table is http://current.geneontology.org/products/upstream_and_raw_data/paint_goa_human.gaf.gz, should this be http://current.geneontology.org/products/upstream_and_raw_data/paint_goa_human_noiea.gaf.gz ? or even http://current.geneontology.org/products/upstream_and_raw_data/paint_goa_human_valid.gaf.gz, not sure if that means the contents of the other files are invalid??

@pgaudet or @thomaspd (and @kltm might be able to clarify what the *_valid.gaf means)

[kltm]

Linked items should be treated as URLs we want to keep permanently--we do not want to start playing routing games. We do not want public pages linking to "upstream_and_raw_data".

I'm a little out of the loop on scope and timeline here. I'm assuming this is part of a larger downloads refactor? For the purposes of drafting, *_valid represents a file that has had the bulk of the gorules applied to it, but has not yet been merged with other files.

[suzialeksander]

@pgaudet can this be added to the Manager's call next week, to let everyone know what's going on? We'll need a lot more of @kltm or other software's time, and probably need to create new files including the split of the uniprot_all.gaf (and new locations??).

[kltm]

Tag back to https://github.com/geneontology/project-management/issues/82#issuecomment-2032996365. TBD, seems like this might be better in that project (or otherwise merged).

[suzialeksander]

I'd like a little more clarification. I agree these are nearly the same thing, but not quite. Are we to have two pages? The other issue outlines a sortable table with annotation counts and one link per species, this is a table with two links.

Is this just a case of the same specs changing over time, or are these actually two different tables?

[suzialeksander]

Discussing this with @pgaudet, if we want to have PAINT annotations as a separate product, we will likely need pipeline refactoring. Upstream products (paint) are likely on a different ontology version than the release product, etc.

[suzialeksander]

Decisions from managers call:

• We should provide the PAINT files, we're OK with these files being a different ontology version
• OK to remove entity types, and we don't really want to provide data that comes from external sources (UniProt)
• Provide what we have, the 150 species is a wishlist goal and we can proceed if we only have the ~20 species. This depends on the separate project to load the data for those extra species, so may not happen soon
• Table types: if we want the smart table (HTML or similar), we would need to bring in Patrick? to make a cleaner table. Other option is a hard coded table, which wouldn't be easily changeable or be able to read metadata. Decision is to have Patrick proceed when we have a spec.
• New table will be directly on https://geneontology.org/docs/download-go-annotations/, http://current.geneontology.org/products/pages/downloads.html will go away
• New spec will be a mock up of table, plus location of files to be added.

[kltm]

@suzialeksander @pgaudet I wanted to check in on the priority of this item and the availability of a spec, as @pkalita-lbl may have some time.