Log paint annotation change counts for each update

[dustine32]

We now have a query/script for generating paint annotation diffs (at least by count) between Panther versions (e.g. 13.1 vs 14.1). I'll need to parameterize this (along with some other changes) for it to plug into the update pipeline.

Also automatically publish these logs to Google drive folder, possibly through an API hook.

[pgaudet]

TABLE 1: Number of nodes in families over the different PANTHER Versions HEADER: PTHR ID | PANTHER version * PTHR ID | Number of nodes | Number of leaves

• File_name = 2019-01-01-Family stats by PANTHER Versions

TABLE 2: IBD count per family

• PTHR ID | Number of IBD
• File_name = 2019-01-01-ibd_count

TABLE 3: IBA count per node

• PTHR ID | PTN ID (that has the corresponding IBD) | GO term | IBA count
• Generate before and after GAFs, compare stats between them
• File_name = 2019-01-01-iba_count

LIST 1: List of PTHR families that had annotations in the previous version and that have 0 annotations in current release

• PTHR ID
• File_name = 2019-01-01_families_lost_all_annotations

LIST 2: Comments and status

• PTHR ID | status | name | creation_date
• File_name = 2019-01-01_curation_status

LIST 3: Remarks

• PTHR ID  |  Comment
• Filename = 2019-01-01 curation_remarks

---

Pascale

[pgaudet]

Please save to: https://drive.google.com/drive/folders/1MrtIQVmtdfd6gJhVcEfofXrU0IIPnOW7

[dustine32]

I have rough draft of the "table 3" report, in two versions, uploaded to the Drive folder: 2019-06-17-iba_count - Lists ancestor nodes and the count diffs of IBA GAF lines derived from them 2019-06-17-iba_count_mods_only - Same as above except that this only counts IBA GAF lines for the 12 MOD organisms:

    "taxon:3702",       # arabidopsis
    "taxon:6239",       # nematode_worm
    "taxon:7955",       # zebrafish
    "taxon:44689",      # dictyostelium
    "taxon:7227",       # fruit_fly
    "taxon:227321",     # aspergillus
    "taxon:83333",      # e_coli
    "taxon:9031",       # chicken
    "taxon:10090",      # mouse
    "taxon:10116",          # rat
    "taxon:559292",     # budding_yeast
    "taxon:284812"      # fission_yeast

A few details of note about these lines:

• Each line has a col for family in case the same PTN node changed families between versions. For 13.1-to-14.1, I don't think these nodes will ever have two differing families. But a node may only be in one of these versions, which would result in a blank Panther ##.# family count col for the version where it's absent.
• If the "after" version family col is blank, this means the PTN was not tracked to the "after" version and therefore all its annotations were obsoleted (or passed on to child nodes). This should correlate with a blank (=0) "Panther ##.#" (in this case, 14.1) col. See for example, PTN002384851->GO:0004930.
• If the "before" version family col is blank, this means the PTN did not have annotations in the "before" version but now does, likely due to inheriting them from a PTN that was its parent in the "before" version but this parent node did not forward track to the "after" version. See for example, PTN001728782->GO:0005886.

[dustine32]

@pgaudet @huaiyumi We found a bug in how the GAFs used for this report were generated. Basically, one of the input file paths was hard-coded to a 13.1-specific node file, which prevented A LOT of 14.1 IBAs from being written. I fixed the file paths and regenerated the 14.1 GAFs and corresponding reports:

2019-06-25-iba_count 2019-06-25-iba_count_mods_only

We should probably delete the 2019-06-17 reports or at least mark them inaccurate.

[dustine32]

@pgaudet For lists 2 and 3 above (curation status and comments), should we limit these lists to only records created/updated during update? The previous curation status list I sent you was the whole table.