NCATSTranslator / Feedback

A repo for tracking gaps in Translator data and finding ways to fill them.
7 stars 0 forks source link

"Streptomycin treats Leukemia" based on bad text-mining results #956

Open khanspers opened 4 hours ago

khanspers commented 4 hours ago

Query: What drugs may treat conditions related to Leukemia?

https://ui.test.transltr.io/results?l=Leukemia&i=MONDO:0005059&t=0&r=0&q=e83261ac-4200-45b4-8d4d-9f0abebefbdc

Streptomycin (an antibiotic) is returned (score of 4.99), from BTE and Unsecret, based on text mining. However, looking at the publication evidence for path a and the supporting paths for path b in the below screenshot, all the papers reference using Streptomycin in culture of cancer cell lines (presumably to avoid contamination), and nothing about the use of streptomycin to treat Leukemia. This can be seen from the "Snippet" that is provided in the UI, but I also opened each paper at PubMed and confirmed this with an in-page search for Streptomycin.

Screen Shot 2024-09-27 at 11 15 44 AM

Similarly, Streptomycin is returned for treats queries for other cancer types, for example lymphoma (score 4.94), renal carcinoma (score 4.94), also with snippets from papers describing cell culture.

Not sure if this is considered a bug or just how text-mining works. But it seems to be different from papers mentioning a drug/chemical in the same sentence as the queried disease, which is what is often in the snippet? Maybe its an ordering/scoring issue?