search

wikipathways / pathway-figure-ocr

Extracting gene sets from published pathway figures
Apache License 2.0
15 stars 2 forks source link

Additional content extraction and annotations from pathway figures #16

Open AlexanderPico opened 4 years ago

AlexanderPico commented 4 years ago

for summarizing query path results during evaluation. For example, annotating pathway figures with terms from the Pathway Ontology will allow BTE users to filter genes by hierarchically-organized pathway classes.

AlexanderPico commented 4 years ago

@ariutta Shall we dive into this one? I was thinking we could add:

  1. PMIDs
  2. Pathway Ontology terms (IDs and names)
  3. Disease Ontology terms (IDs and names)

You have 1 already. We have the start of 3 (from Jensen). And we should brainstorm on 2.

Other ideas?

AlexanderPico commented 4 years ago

@khanspers Why don't you try getting GO:BP terms associated with PFOCR results.

AlexanderPico commented 4 years ago

@ariutta I'll send you a tsv with figid mappings to disease terms. Can you work with kevin to get consensus on the formatting.

ariutta commented 4 years ago

@kevinxin90, we're looking to add more fields (PMID, Pathway Ontology, Disease Ontology) to our data for BTE. What do you think about the format below (the values are just placeholders)? The keys match http://identifiers.org/ prefixes in order to ensure they stay consistent and unique.

{"_id": "PMC0000000__nihm00000000",
 "associatedWith": {
    "figureUrl": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC0000000/bin/nihm00000000.jpg",
    "pmc": "PMC0000000",
    "pubmed": "00000000",
    "ncbigene": ["1000", "1001", "1002", "1003"],
    "PW": ["0000001", "0000002"],
    "DOID": ["10000", "10001"]
  }
}
kevinxin90 commented 4 years ago

@ariutta Hi Anders, this looks great to me!!

AlexanderPico commented 4 years ago

@ariutta, Here's a map of figid to doid and human readable terms: https://www.dropbox.com/s/vgazcxdq4wsl5yk/pfocr_disease_map.tsv?dl=0

@kevinxin90, Should we include both DOID and term name? As a proposed "node property" the human readable term name may be more useful than DOID. But if you already have a mapping for Disease Ontology IDs and terms, then it's perhaps not necessary. Thoughts?

kevinxin90 commented 4 years ago

@AlexanderPico Yes. I think it's a good idea to include both DOID and term name in the API response. For Biothing Explorer, it doesn't matter whether you include the name or not (we will do the ID resolving internally). But as an API itself, it would be very useful (in case the user doesn't access it through BioThings Explorer).

ariutta commented 3 years ago

@kevinxin90, in this reporting period, @AlexanderPico and I have done more work to parse our PFOCR data, using APIs like PubTator to extract chemical and disease mentions in the OCR text. Notice that these disease mentions are pulled directly from the OCR text, unlike the disease associations Alex referenced above, which come from pathway enrichment analysis. I'm thinking of the former as disease mentions and the latter as disease annotations.

In order to include this additional information, I propose we use the following updated JSON format, but I'm open to suggestions. We also need to distinguish between mentions vs. annotations somehow, so I set the mentions as key/value pairs under associatedWith, and the annotations as key/value pairs under associatedWith.annotations.

{
  "_id": "PMC0000000__nihm00000000",
  "associatedWith": {
    "figureUrl": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC0000000/bin/nihm00000000.jpg",
    "pmc": "PMC0000000",
    "pubmed": "00000000",
    "ncbigene": [
      "1000",
      "1001",
      "1002",
      "1003"
    ],
    "disease:mesh": [
      "D000860"
    ],
    "chemical:mesh": [
      "D000431",
      "D000079",
      "D000085"
    ],
    "annotations": {
      "PW": [
        "0000001",
        "0000002"
      ],
      "DOID": [
        "10000",
        "10001"
      ]
    }
  }
}

(dummy data -- just showing the format)

Questions:

  1. How do you want to distinguish mentions vs. annotations?
  2. The PubTator API returns MESH IDs, so that's why I used MESH IDs for disease:mesh but not DOID. Would it be worth translating to use a single identifier for diseases?
  3. Unlike the other identifiers, mesh covers genes, diseases, chemicals, etc. That's why I specified disease:mesh. Do you prefer something else?
ariutta commented 3 years ago

@andrewsu, you're also welcome to comment on this format, if you're interested, especially how to distinguish mentions vs. annotations.

kevinxin90 commented 3 years ago

Hi @ariutta, thanks for the update. Here're a few suggestions from me.

  1. Group the mentions and annotations as two separate dictionaries.
  2. Group the results based on the semantic type, e.g. gene, chemical, disease
  3. It's fine to keep mesh for results from pubtator

Here is what I would propose:

{
  "_id": "PMC0000000__nihm00000000",
  "associatedWith": {
    "figureUrl": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC0000000/bin/nihm00000000.jpg",
    "pmc": "PMC0000000",
    "pubmed": "00000000",
    "annotations": {
        "pathway": {
            "PW": [00001, 00002],
        },
        "disease": {
             "DOID": ["DOID:0001", "DOID:000012"]
        }
    },
    "mentions": {
        "disease": {
              "mesh": "D000860"
        },
        "chemical": {
               "mesh": ["D0001", "D00002"]
        }
     }
}
ariutta commented 3 years ago

@kevinxin90, that looks great. I just need to get you an updated file with this format.

ariutta commented 3 years ago

Hi @kevinxin90, I wanted to let you know I have the chemicals extracted from almost all the PFOCR pathway figures (63591 of 64643 because the PubTator API returned an error for some of them). I currently have them in TSV format (link included in case you want to take a sneak preview), but I'll get an updated file with the format we agreed on.

Summary Stats

All pathway figures

Just the pathway figures with 10+ genes:

ariutta commented 3 years ago

@kevinxin90,

Here is an updated file in the format we discussed, including chemicals in addition to genes this time: https://www.dropbox.com/s/m03hd447oi3yjz1/pfocr_biothings_65k_20201203.ndjson?dl=0

Summary Stats

for pathway figures having at least 3 chemicals and/or genes found

Questions:

  1. For identifiers that could be represented as integers, I'm still formatting them as strings, e.g., NCBI Gene or PubMed PMID. Let me know if you want them to be integers.
  2. For cases where a figure had genes found but no chemicals (or vice versa), I still created an empty array [] for mentions.genes.ncbigene or mentions.chemicals.mesh. Let me know if you'd prefer those items to just not be included at all.
kevinxin90 commented 3 years ago

@ariutta I found a couple places where the value of pubmed equal to 0. Does it mean pubmed id is not found? Example:

'_id': 'PMC6887797__10.1177_1758835919887665-fig1.jpg', 
'associatedWith': {
    'figureUrl': 'https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6887797/bin/10.1177_1758835919887665-fig1.jpg', 
    'pmc': 'PMC6887797', 
    'pubmed': 0, 
    'mentions': {
        'chemicals': {
            'mesh': ['D004967']
        },
        'genes': {
            'ncbigene': ['10000', '1019', '1021', '144455', '1869', '1870', '1871', '1874', '1875', '1876', '1956', '2064', '2065', '207', '208', '2475', '595', '6198', '6199', '7248', '7249', '79733']
         }
       }
    }
}
ariutta commented 3 years ago

Good catch! Yes, you're right. I'll take a look at filling those in.

kevinxin90 commented 3 years ago

Great! I currently just removed those pubmed fields if they equal to 0.

@ariutta The new API is currently up at: https://biothings.ncats.io/pfocr

Some example queries:

https://biothings.ncats.io/pfocr/geneset/PMC4671449__medscimonit-21-3736-g001.jpg

Query by Chemical MESH ID: https://biothings.ncats.io/pfocr/query?q=associatedWith.mentions.chemicals.mesh:D000431

Query By Gene NCBIGene ID: https://biothings.ncats.io/pfocr/query?q=associatedWith.mentions.genes.ncbigene:1017

AlexanderPico commented 3 years ago

@kevinxin90 Do the chemical MeSH IDs connect in BTE to any other entries, e.g., pharmaceutical products? It would be nice to mention a query path in the annual report that is enabled by this new deposition of chemicals in pathway figures.

AlexanderPico commented 3 years ago

@kevinxin90 Is there a particular place we should upload future .ndjson files, i.e., somewhere that is periodically checked and queued up for BTE integration? Or is this something we should work out next year?

kevinxin90 commented 3 years ago

@AlexanderPico Hi Alex, if it's small. You could just upload to this github. For us, we only need a static url we can track the last-modified date. It could also be a google cloud storage or other cloud storage services. You could view an example here. The example I showed above, if you look at line 19, we set a cron job to check for update every Monday, and that one is hosted on Google Cloud API.

kevinxin90 commented 3 years ago

pharmaceutical products

@kevinxin90 Do the chemical MeSH IDs connect in BTE to any other entries, e.g., pharmaceutical products? It would be nice to mention a query path in the annual report that is enabled by this new deposition of chemicals in pathway figures.

We don't really distinguish pharmaceutical products (I assume drug, right?) and ChemicalSubstance in BTE. But we do can do some ID translation (e.g. MeSH -> Drugbank ID) within BTE.

We can make a query path like Disease -> Gene -> Chemical (with the later step using PFOCR as one of the consulted resources? since PFOCR has information about what genes and chemicals are in the same pathway) Up for better suggestions.

AlexanderPico commented 2 years ago

@ariutta Should this be closed? Are chemical, disease ont and pathway ont terms integrated?

ariutta commented 2 years ago

Chemicals: yes. Diseases, pathways or amino acids: I'm not sure.

AlexanderPico commented 2 years ago

Next, let's add title and disease mentions like so:

{
  "_id": "PMC0000000__nihm00000000",
  "associatedWith": {
    "title": "signaling in immune response",
    "figureUrl": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC0000000/bin/nihm00000000.jpg",
    "pmc": "PMC0000000",
    "pubmed": "00000000",
    "mentions": {
        "diseases": {
              "mesh": ["D000860", "D000333"]
        },
        "chemicals": {
               "mesh": ["D0001", "D00002"]
        },
        "genes": {
               "ncbigene": ["1234", "2345"]
        }
     }
}
ariutta commented 2 years ago

How inclusive should we be for chemicals? For example, some of our collaborators don't want side metabolites included in their analyses, e.g., Na+, NADPH or S-adenosyl-L-methioninate.

Thanks to @tokebe's suggestion, I ran a sample of the PFOCR chemicals results through the node normalizer. I got back the following:

biolink types

normalized identifier namespaces

I'm proposing that we go ahead and incorporate all the results into BTE, without any filtering to exclude items like side metabolites. However, if it's preferable, I could split up the chemicals into biolink categories.

andrewsu commented 2 years ago

I'm proposing that we go ahead and incorporate all the results into BTE, without any filtering to exclude items like side metabolites.

I agree with this strategy!

ariutta commented 2 years ago

I've got a draft version of the latest export file ready: https://github.com/wikipathways/pfocr-pipeline/raw/main/export/bte_chemicals_diseases_genes.ndjson

Note this version does not have PubMed IDs. Is it important that I provide them, or is this something easily handled by existing TRAPI APIs like node normalizer?

andrewsu commented 2 years ago

On quick glance, looks good! I do not believe that Translator has an easy mechanism to translate PMCIDs to PMIDs. So if it's easy for you to add, please do. If not, the PMCID will be fine as provenance for now...

ariutta commented 2 years ago

@andrewsu, we have an updated dataset available here: https://www.dropbox.com/s/1f14t5zaseocyg6/bte_chemicals_diseases_genes.ndjson?dl=0

This dataset is formatted the same way as the previous one, except:

We now have chemicals, diseases and genes. The new genes are from our latest batch of PFOCR data. The new chemicals are from both the latest batch of PFOCR data as well as from relaxing the filtering restrictions we applied last time -- we've included all chemicals except ones with names only containing the letters HCONSP because those ones introduced too many false positives.

Summary stats: