NCATSTranslator / Feedback

A repo for tracking gaps in Translator data and finding ways to fill them.
7 stars 0 forks source link

Missing Knowledge Level #818

Closed cbizon closed 2 months ago

cbizon commented 3 months ago

Query: Chemicals to decrease JAK1. PK: a5731faf-7cd8-42f2-81ef-5cf6030931fa

Top answer is Tofacitinib. This is a lookup result. But there are two lookups: image

I think that this is because one of the two ARAs responding is not including Knowledge Levels. It's either ARAX or Unsecret, both tagged.

sstemann commented 3 months ago

i think it may actually be that ARAX responds with 3 edges like this:

image

and 3 edges like this:

image

you would have to hover over the edge buttons to see the difference in predicates

cbizon commented 3 months ago

That is odd, because I don't know where those activity_or_abundence edges are going. But the UI is only showing causes decreased activity of for both of those lookup paths. And the first one doesn't have either a badge or a robot. And if I click on the one without the badge it shows that there is no KL:

image

kaiwenho commented 3 months ago

The knowledge graphs ingested by Unsecret do not contain KL and AT information in the edges. The unknown KL is from the answers returned by Unsecret. I will check for updates from the KPs to see if they now include this information and are available in KGX format.

dkoslicki commented 3 months ago

RTX-KG2 will have KL/AT on all edges in version KG2.10.0 whose release is on the horizon; should be fixed in ARAX then

saramsey commented 2 months ago

@dkoslicki is correct-- the code for KL/AT has been added to RTX-KG2.10.0, which we are aiming to get into Sprint 4 (though it will be down to the wire).

saramsey commented 2 months ago

See these issues to track our team's progress toward providing KL/AT in the RTX-KG2 KP: https://github.com/RTXteam/RTX-KG2/issues/358 https://github.com/RTXteam/RTX/issues/2319

My understanding from @ecwood is that the code updates for providing KL/AT edge attributes in the KG2pre graph build are already completed. Now the downstream work (@amykglen and @sundareswarpullela can comment) to provide those edge attributes in KG2c and in responses returned by the RTX-KG2 KP, is happening. We are aiming to roll this out in Sprint 4 (7/26 release-to-test deadline).

saramsey commented 2 months ago

Confirmed, this is fixed in Translator CI

Screenshot 2024-07-28 at 3 27 37 PM

saramsey commented 2 months ago

This is not yet fixed in ITRB TEST, because there was an unrelated "package dependency" hiccup in the deployment that we requested on 7/19. Our team has requested (on 7/25) deployment of a patched release to ITRB TEST, and when that rolls out, the fix for this issue should be in ITRB TEST.

I will verify it by posting to kg2.test.transltr.io this query graph:

{
  "edges": {
    "t_edge": {
      "attribute_constraints": [],
      "knowledge_type": "inferred",
      "object": "on",
      "predicates": [
        "biolink:affects"
      ],
      "qualifier_constraints": [
        {
          "qualifier_set": [
            {
              "qualifier_type_id": "biolink:object_aspect_qualifier",
              "qualifier_value": "activity_or_abundance"
            },
            {
              "qualifier_type_id": "biolink:object_direction_qualifier",
              "qualifier_value": "decreased"
            }
          ]
        }
      ],
      "subject": "sn"
    }
  },
  "nodes": {
    "on": {
      "categories": [
        "biolink:Gene"
      ],
      "constraints": [],
      "ids": [
        "NCBIGene:3716"
      ],
      "is_set": false,
      "set_interpretation": "BATCH"
    },
    "sn": {
      "categories": [
        "biolink:ChemicalEntity"
      ],
      "constraints": [],
      "is_set": false,
      "set_interpretation": "BATCH"
    }
  }
}

and inspecting the edge attribute list for the tofactinib result.

saramsey commented 2 months ago

OK, I am now seeing KL and AT edge attributes on results from kg2.test.transltr.io, as of a few minutes ago:

Screenshot 2024-07-30 at 3 18 28 PM

edeutsch commented 2 months ago

@saramsey Is there a reasonably high level overview of how KL and AT and treats are implemented in KG2.10? I feel a little in the dark about how these are set and would like to understand a bit better.

ecwood commented 2 months ago

@edeutsch In KG2.10.0, we are using the long-term specifications discussed here: https://github.com/NCATSTranslator/ReasonerAPI/blob/master/ImplementationGuidance/Specifications/knowledge_level_agent_type_specification.md#long-term-specification. Our mappings by each edge source are listed here: https://github.com/RTXteam/RTX-KG2/blob/master/maps/knowledge-level-agent-type-map.yaml.

edeutsch commented 2 months ago

Thanks @ecwood that helps a bit. 100s of lines of YAML isn't the easiest to digest, but I think I got it mostly. Note that it looks like the intent for text mining providers was to leave KL as "not provided" instead of "prediction": https://docs.google.com/document/d/140dtM5CjWM97JiBRdAmDT-9IKqHoOj-xbE_5TWkdYqg/edit#heading=h.kajf0wetgvg6

saramsey commented 1 month ago

@edeutsch yes, @ecwood has notes from our meeting with Matt Brush and they reported that—based on their notes—text-mined triples should have a "knowledge leve." of "not provided".