NCATSTranslator / Feedback

A repo for tracking gaps in Translator data and finding ways to fill them.
7 stars 0 forks source link

Weird path logic for MVP1 #465

Open khanspers opened 1 year ago

khanspers commented 1 year ago

Query: What drugs may treat Xeroderma Pigmentosum? https://ui.test.transltr.io/results?l=Xeroderma%20Pigmentosum&i=MONDO:0019600&t=0&q=4ce93edf-ee2b-4ce4-99cd-a8e03e85d5ff

Cadmium and Mercury are returned in the top 10, and the paths are similar in both cases (see screenshots), for example:

Cadmium - has_decreased_uptake_caused_by - Slc30a1 - causes_decreased_transport_of - Zinc - treats - Xeroderma Pigmentosum

Screen Shot 2023-08-09 at 11 11 31 AM Screen Shot 2023-08-09 at 11 11 23 AM

This makes no sense, there is no effect on Slc30a1 by Cadmium, its the opposite.

sierra-moxon commented 1 year ago

UI "Reasoning Agent" is ROBOKOP, but if I dive into the predicates, I also see TextMiningProvider in the sources.

sandrine-m commented 11 months ago

RETESTED today and could confirm the offending results are still showing up from ARAGORN. I am still waiting for ARAGORN to complete.... It has been quite a long time though... I will stop there, this issue will need more retesting.

sandrine-muller-research commented 11 months ago

Retested today: cannot reproduce on PROD, nor on TEST, nor on CI. I went ahead and posted this specific query to ARAX UI (CI environment):

{
  "edges": {
    "t_edge": {
      "attribute_constraints": [],
      "knowledge_type": "inferred",
      "object": "on",
      "predicates": [
        "biolink:treats"
      ],
      "qualifier_constraints": [],
      "subject": "sn"
    }
  },
  "nodes": {
    "on": {
      "categories": [
        "biolink:Disease"
      ],
      "constraints": [],
      "ids": [
        "MONDO:0019600"
      ],
      "is_set": false
    },
    "sn": {
      "categories": [
        "biolink:SmallMolecule"
      ],
      "constraints": [],
      "ids": [
        "PUBCHEM.COMPOUND:23931"
      ],
      "is_set": false
    }
  }
}

And I am not able to retrieve any path like that... However, both Unsecret and Aragorn result in a 598 so perhaps somebody could retest later?

sandrine-muller-research commented 11 months ago

retested in TEST

khanspers commented 2 months ago

Retested this in test today: https://ui.test.transltr.io/main/results?l=Xeroderma%20Pigmentosum&i=MONDO:0019600&t=0&r=0&q=56804803-958c-46d7-b91d-068c1e9f97ac

Cadmium and Mercury are still returned, both direct lookup edges, although with lower scores than the original query for this ticket. From Aragorn/Robokop.

Screen Shot 2024-07-08 at 2 50 00 PM

The weird thing is that in ARAX both results look like this:

Screen Shot 2024-07-08 at 2 50 40 PM

I've never seen that before. Support graph:

Screen Shot 2024-07-08 at 2 50 47 PM

All the edges are "occurs in literature with". Not sure how that becomes "treats" but it seems wrong.

cbizon commented 2 months ago

Needs a FUGU retest

sandrine-muller-research commented 1 month ago

what does FUGU means? Here a retest: restested today on test instance: PK: 93306977-7e90-4f88-ba65-d1a99bdf07d6

sstemann commented 1 month ago

@cbizon this is still an issue in Fugu Test PK: 6a96fc3a-2127-4b9c-8763-cb811985b169

For both Mercury(0) and Cadmium. Both come through to the UI as one-hop, with the only EPC being ARAGORN. Total ARAGORN results in the Test UI 429.

there are significanly less results (total 101) from ARAGORN in CI (Fugu+) https://ui.ci.transltr.io/main/results?l=Xeroderma%20Pigmentosum&i=MONDO:0019600&t=0&r=0&q=4674bef4-ee4e-4f8a-965e-56c89652a1e0

In CI, this query also returns MANY paths with pathwhiz nodes, we dont see this behavior in Test.

Is there a fix for the original issue that will be pushed to Test soon? if not should the label update to Guppy?

cbizon commented 1 month ago

@sstemann we had a cache in TEST that didn't get cleared when we pushed FUGU so there were still EEL answers in there. Max has cleared the cache now and we should see FUGU answers. For instance, in this PK: bf85c628-5b0b-4966-b5e9-67a7cc089820 that I just ran, there is no Cadmium or Mercury answer.

There were multiple fixes here, one that I'm pretty sure solved the missing edge problem, but also tweaks to the rules that removed cadmium and mercury.

Either way, if this looks good to you, I think we can close this one.

sstemann commented 1 month ago

it looks like Cadmium and Mercury are gone but I think this reasoning path that @khanspers raised is still there, and I'm not sure if thats what was intended to be updated @cbizon ?

https://ui.test.transltr.io/main/results?l=Xeroderma%20Pigmentosum&i=MONDO:0019600&t=0&r=0&q=689fdfb0-b2fa-411e-8a4c-34ef879a1787

https://arax.ci.transltr.io/?r=689fdfb0-b2fa-411e-8a4c-34ef879a1787

Cisplatin is Scored 5.0

image

These paths are from ARAGORN image

cbizon commented 1 month ago

I'm going to review this rule, and see whether or not we want to keep it. For Guppy. The filtering has removed cadmium and mercury as results.

cbizon commented 2 weeks ago

Discussed how to handle on ARAGORN 8/30. We are going to add some deny list to certain rule edges for this, and also experiment with some updates to the rule