Open saramsey opened 2 months ago
I'm not sure how the rank is calculated other than its from Appraiser. here is the backend O&O components from Appraiser malignant pancreatic neoplasm_test_2024_7-8_14_53_3972dc9b-ca4b-4065-aa6c-70b795593d75.xlsx in the flattened Excel. Asking O&O to take a look.
subjectNode_name | interleukin-1, beta UMLS:C0021753 | nivolumab UNII:31YO63LBSN | cetuximab PUBCHEM.CPD:14122979 | ramucirumab UNII:D99YVK4L0X |
---|---|---|---|---|
rank | 1 | 2 | 3 | 4 |
sugeno_score | 1 | 1 | 1 | 1 |
comp_confidence_score | 1 | .97 | .97 | .86 |
comp_novelty_score | 0 | 0 | 0 | 0 |
comp_clinical_evidence_score | 0 | 1 | 1 | 1 |
weighted_mean_score | 0.48 | 0.94 | 0.94 | 0.88 |
normalized_score | 100 | 81.2 | 80.8 | 29 |
ARAX_score | 0 | 0 | 0 | 0 |
unsecret_score | 1 | 0 | 0 | 0 |
improving_agent_score | 0 | 0.36 | 0.37 | 0.4 |
biothings_explorer_score | 0 | 0.95 | 0.95 | 0.76 |
aragorn_score | 0 | 0 | 0 | 0 |
there are two factors here: why did unsecret rate this so high, and whether there are enough pathological cases where one ARA thinks so highly that the confidence (and thus sugeno) scores are distorted. Assigning primarily to @webyrd at this point.
There is a bunch of issues of this nature that have been filed lately, and this can be one datapoint in informing @dkoslicki @Rosinaweber .
Like the previous, we are examining these and will make sure these can all be fixed still in this phase.
In the ui.test.transltr.io system, just now, I ran a test query "What drugs may treat conditions related to malignant pancreatic neoplasm?".
Here is a link to the results: https://ui.test.transltr.io/main/results?l=Malignant%20Pancreatic%20Neoplasm&i=MONDO:0009831&t=0&r=0&q=3972dc9b-ca4b-4065-aa6c-70b795593d75
The first result, "Interleukin 1, Beta", is scored 5 out of 5 and has the following two explanatory graphs:
These paths don't seem to justify a score of 5 out of 5 for this result. I note that in a rodent study, overexpression if IL1beta induces gastric inflammation and cancer: https://www.nature.com/articles/s41568-021-00363-z
Of course, neither that study nor the two above result graphs mean for certain that IL1-beta is wrong as a result for the query, but I'm respectfully questioning the 5 out of 5 score for this result.