Normalize Evidence Direction

[bwalsh]

[grmayfie]

This is not 'evidence direction' as we had discussed. What is this actually referring to?

[bwalsh]

From https://civic.genome.wustl.edu/faq

What is the difference between "Supports a Negative association" and "Does not support a Positive association"? This can be confusing. We have reserved "Does not support - Positive" for statements that contradict previous statements that are supporting positive associations. For example, they would read "Contrary to the previous study which found this mutation sensitive to drug X, this study reported no effect." The information that this study "Does not support" the prior study's conclusion is what we are trying to capture with these classifications.

[grmayfie]

This sounds like clinical significance rather tan evidence direction; perhaps these titles got swapped on accident?

[grmayfie]

I don't think there's anything I've seen in any of the other sources we're pulling from that looks anything like this, and so I don't believe there's anything to normalize on. None of the other sources make 'supporting' or 'not supporting' editorials.

Suggest we kick to the VICC group to see what they want to do with this.

[ahwagner]

Hi @mayfielg, @bwalsh. Just to clarify, these values are all part of the CIViC evidence type statements. The primary evidence type field is Predictive/Diagnostic/Prognostic, and the evidence direction is supports/does not support. Clinical significance is a third part of the evidence type statement which specifies sensitivity/resistance, positive/negative, good/poor outcome, pathogenic/benign, depending on the evidence type.

A table of how these fields are used collectively is available with supporting examples: https://civic.genome.wustl.edu/help/evidence/evidence-types

[grmayfie]

Thanks for the clarification. That chart definitely explains a lot better what's happening with the different CIVIC tags.

I think though that we do still need to have a conversation about what to do with 'supports' vesus 'does not support' at the VICC group level, though, since CIVIC is the only source that differentiates their data that way.

[malachig]

I do think this is worthy of further discussion. It would be a bit surprising if negative results are not included in any of the other resources. Consider the following (somewhat contrived) scenario:

• Group A conducts a large clinical trial and finds that BRAF V600E +ve melanoma treated with dabrafenib have improved survival compared to mutation -ve patients (p-value = 0.03). The authors conclude that one should consider mutation status in selecting this drug.

• Group B conducts a large clinical trial and finds no significant difference in response to dabrafenib between the BRAF V600E +ve and -ve melanoma patients (p-value = 0.43). The authors conclude that one should NOT consider mutation status in selecting this drug.

Both studies are asking whether presence of a variant (BRAF V600E) predicts response to a drug (dabrafenib).

The purpose of "Supports" and "Does not support" is to allow both of these findings to be captured as evidence records. As the evidence gathers one can assess where there are conflicting findings when proposing to a final assertion.

If only CIViC has this concept, the simplest path forward is to simply drop all the "Does not support" evidence (it is in the minority).

You can see a breakdown here: https://civic.genome.wustl.edu/statistics/evidence

Currently there are 168 "Does not support" evidence records. Quite a few, but this represents only 4% of all evidence records.

[grmayfie]

BW & GM: Deferred until feature interface design as a potential search parameter

[jgoecks]

@mayfielg @bwalsh I do not think we should defer this. Many evidence records in several databases have negative directions, and postive/negative direction has implications for analysis of TCGA and GENIE.