Posting feedback from meeting with Paul Watkins, SME, July 16, 2024. Note that Paul has been engaged with Translator from nearly the beginning of the program, so some of his feedback reflects that experience. I still need to unpack this issue, but I'm posting it just so I don't lose Paul's feedback. We tested in TEST (eel).
We started off our meeting with me giving Paul a quick demo of the UI, primarily to set expectations. He had seen a demonstration version but not the alpha public release. Interestingly, Paul's first reaction was "What is the purpose of Translator?" I think his reaction was based largely on the fact that there were only five available questions. He particularly found the four MVP2 questions to be of limited value, in terms of the types of questions he was interested in. (See more below.)
After my quick demo, Paul took over. He jumped around a lot during the session, in part because I think he was a bit frustrated, but I captured three PKs.
The first query was for MVP1. Paul typed in “hepatosteatosis”, which was not found. His natural instinct was to assume the concept/disease simply wasn’t available. He needed prompting to search for a synonym, e.g., “hepatic steatosis”. This might be worth addressing in the FAQ page or somewhere. [Note that this was not the best way to start the session!]
Paul found the scoring to be very confusing. I pointed out the definition, but he was not satisfied, as it wasn't sufficiently detailed. [Note that no one every notices the little information button over "SCORE". Not sure why.]
The first set of answers that were returned were all scored very low (<1.0).
The complete answer set had most answers scored 5.0; in fact, Paul needed to jump many pages in order to find any answers with scores <5.0.
Paul found the staggered responses and "load new results" to be more confusing than helpful.
“System is very slow.”
Paul did not find the filters to be helpful.
Paul also did not find the “graph view” to be helpful, but rather confusing.
He did find the Text Mining KP wiki page to be helpful in interpreting the results. He also found the publications and snippets to be helpful.
Paul found the MVP2 questions confusing and very restrictive.
He would have preferred user-directed, iterative querying.
“My GWAS identified susceptibility genes for DILI. I want to know why. [Translator] answers are too simple. Mostly one hop, two hop. What happened to the more complex results, graph networks?" [Not an exact quote, but close.]
Toward the end of the session, I brought up the MCQ and Pathfinder queries and showed him some examples. Paul definitely found these capabilities to be more interesting than the MVP1 or MVP2 queries (and especially the MVP2 queries), which is encouraging.
Posting feedback from meeting with Paul Watkins, SME, July 16, 2024. Note that Paul has been engaged with Translator from nearly the beginning of the program, so some of his feedback reflects that experience. I still need to unpack this issue, but I'm posting it just so I don't lose Paul's feedback. We tested in TEST (eel).
We started off our meeting with me giving Paul a quick demo of the UI, primarily to set expectations. He had seen a demonstration version but not the alpha public release. Interestingly, Paul's first reaction was "What is the purpose of Translator?" I think his reaction was based largely on the fact that there were only five available questions. He particularly found the four MVP2 questions to be of limited value, in terms of the types of questions he was interested in. (See more below.)
After my quick demo, Paul took over. He jumped around a lot during the session, in part because I think he was a bit frustrated, but I captured three PKs.
The first query was for MVP1. Paul typed in “hepatosteatosis”, which was not found. His natural instinct was to assume the concept/disease simply wasn’t available. He needed prompting to search for a synonym, e.g., “hepatic steatosis”. This might be worth addressing in the FAQ page or somewhere. [Note that this was not the best way to start the session!]
https://ui.test.transltr.io/main/results?l=Prostate%20Cancer&i=MONDO:0008315&t=0&r=0&q=94488aa3-66a2-4d62-86fc-00e2a540da0a
https://ui.test.transltr.io/main/results?l=Xylazine&i=CHEBI:92386&t=4&r=0&q=8263430c-667c-4e88-83e5-03fdcb6d7408
https://ui.test.transltr.io/main/results?l=EXOC3L4%20(Human)&i=NCBIGene:91828&t=1&r=0&q=7576a180-2500-43d3-b3e8-9133f0ff9d0f
Toward the end of the session, I brought up the MCQ and Pathfinder queries and showed him some examples. Paul definitely found these capabilities to be more interesting than the MVP1 or MVP2 queries (and especially the MVP2 queries), which is encouraging.