Update Treats to May Treats in the UI

[sstemann]

Based on Alumni Feedback at the September 2023 Relay, the "treats" predicate should read "may treats" in the UI for every instance of Treats.

[andrewsu]

The alumni feedback is invaluable of course, but I'd suggest we not immediately implement this suggestion as stated though. I think this exact idea has been discussed (though I'm forgetting in which call). But we instead decided that it would be better for the UI to display visual indicators of EPC (including the enhancements for knowledge level and agent type). We felt that those steps (*) would better convey to the user the strength of the evidence, rather than just uniformly watering down the treats predicate everywhere.

(*) While I'm happy to leave the actual implementation of this to the UI team guided by user testing, just noting that I don't think we discussed the specific things the UI team was planning in this regard.

[gglusman]

I quite disagree with that. I think it would be best to reserve the clear-cut 'X treats Y' statement for, well, when it's clear-cut. It probably wouldn't be too hard to implement a dynamic display based on confidence level, selecting a predicate to be displayed from among 'treats', 'very likely treats', 'likely treats', 'may treat', etc. The point is that 'X treats Y' is a very strong statement that the user immediately gets, way before they dig into the EPC (if they do at all) to discover that this may actually be a very low confidence tenuous inference from barely supported assertions that need to be independently verified.

[andrewsu]

I quite disagree with that. I think it would be best to reserve the clear-cut 'X treats Y' statement for, well, when it's clear-cut.

@gglusman I disagree that we disagree... ;) My reading of the issue description is that all "treats" statements would be watered down to something less definitive, leaving no option to use "treats" even for clear-cut cases. I agree that the UI somehow needs to dynamically convey the confidence behind an edge or a result based on the specifics of that edge/result. I'm not passionate about the specific mechanism about conveying that e.g., "'treats', 'very likely treats', 'likely treats', 'may treat', etc.", but aside from that minor point I agree with everything you wrote...

[gglusman]

Fair! But: if only one static predicate will be used, I think it should be 'may treat', not 'treats'.

[codewarrior2000]

Speaking of static, the dichotomy between 'may treat' and 'treats' is wispy and ephemeral and at the mercy of external agencies. Take phenylephrine as an example. Just before the Relay we thought it treats nasal congestion. And now we think it doesn't, according to the FDA.

Anyway, @andrewsu's notion to leave well enough alone and not make an across the board refactoring from "treats" to "may treats" (but to let EPC be the guidance to users about "treats") makes good engineering sense. After all, think of the enormous regression testing that needs to be done after the refactoring.

[gglusman]

This is kind of rehashing previous discussions, but might as well document them here as well.

The proposal is NOT to refactor the predicates and the EPC: it is only a change in what is being displayed. Nothing needs to change 'under the hood'; it's simply the recognition that presenting the results as assertions of 'X treats Y' is simply incorrect. Some of us have raised the point before, and we now heard the same from alumni.

It would be better for Translator to underpromise and overdeliver (i.e., have the user rejoice over an amazing result that was cautiously presented as a hypothesis) than the opposite (i.e., have the user dismiss Translator as overselling weak inferences as facts).

[codewarrior2000]

No one said proposal is to refactor the predicates and the EPC. I thought the discussion was strictly about the UI layer.

[dnsmith124]

One could argue that if the underlying data is providing results that are incorrect enough to require the UI to reword them, then maybe that result should not exist in the data (or should use a different predicate).

I'm also of the mind that it's dangerous for the UI to start modifying the names of predicates.

[gglusman]

After reviewing the current UI status, I think the apparent disagreement in this thread may be entirely caused by me talking about a different part of the UI than y'all.

To review: 1) Currently we have the button to 'explore the relationship between drug and disease', which doesn't talk about 'treats' anymore, so no issue here. 2) While computing/loading results, the text displayed is: "Showing results for: What drugs may treat: [disease/condition]". 3) For each result in the list, the text displayed is: "[result name] [N] Paths that may treat [disease/condition]". 4) When showing the paths themselves, an actual 'Treats' predicate may be displayed. 5) After clicking on the 'treats' icon, the text displayed is: "Showing Evidence for: [result name] Treats [disease/condition]", and in the 'Sources' and 'Publications' tables underneath, "[result name] treats [disease/condition]".

My initial understanding of this ticket was that the proposal was to make whatever changes were needed to make all five levels display 'may treat'. I then (mis)interpreted the pushback to mean all five should show 'treats'. I think it's perfectly correct to keep showing 'may treat' in (2) and (3), and to keep 'treats' in (4) and (5) as they are reflections of the underlying predicate, and are tightly linked to statements of evidence: (4) instructs the user to clink to get evidence, while (5) is shown in the actual evidence context.

[codewarrior2000]

@dnsmith124 Right! The reality about the vast corpus of knowledge in Translator is that it occupies a broad spectrum, from truthfulness to truthiness. If I understand @andrewsu's point, the system will depend on the "UI to display visual indicators of EPC (including the enhancements for knowledge level and agent type)", which will give the user an idea about the underlying data.

So yes, it's superfluous for the UI to start modifying the names of predicates

[codewarrior2000]

@gglusman Excellent summation. No one could have written it better than that.

[sierra-moxon]

I'm also going to document here, that we have had many conversations about the treats predicate and its problematic semantics (beyond just the "may", "might" epistemic qualifiers to this predicate). We really need to both characterize the confidence, and also have more predicates that allow us to be more specific about the statements we're making.

[dnsmith124]

@gglusman I completely agree, thank you for clearing the fog on this! @codewarrior2000 Exactly, I think our upcoming changes that visually separate inferred edges from lookups should address the root cause of this issue

[mbrush]

It sounds then like the only place treats shows up as a predicate then is deeper in the weeds of the support paths, and that there is general agreement that it is OK to keep this as is for now - given anticipated enhancements to come. On this front, I see three key efforts that will address concerns the alumni or others have about 'over-promising' with 'treats' at this level in the display:

1. The addition of knowledge level/agent type tags in the UI (and possible adjustment of the predicate text in the UI to reflect confidence based on these tags, e.g. treats --> may treat if the knowledge level = prediction)
2. The upcoming refactor to how we represent treatment relationships in the underlying data - which will likely include use of more precise/conservative predicates in cases where 'treats' is currently 'overpromising'.
3. A broader re-factor to the layout of results, "answer edges", and support paths in the UI, that will allow for a clearer understanding of the relationships between these levels of information, and provide additional ways / points where we can clarify the strength of any 'treats' edges.

I think all these efforts are relatively high priority - and their implementation should be coordinated to be sure they address the key concerns raised in this ticket.

[gprice1129]

@mbrush The UI team is currently working (design and implementation work being done) on how to incorporate knowledge level into the UI with the aim of making:

1. The distinction clear between some of the knowledge levels.
2. The relationship between inferred edges and supporting edges that support those inferred edges very clear to users.

There will most likely be other changes in the future regarding knowledge level and agent type but we don't want to bite off too much at one time!

[mbrush]

Makes sense @gprice1129 . . . I expanded a bit on these ideas in my comment on #552 - but these are just things to be thinking about as initial small scale improvements are made.

[sstemann]

this is updated in the UI.