Separate clinical significance from confidence in Pathogenicity classification predicates

[mbrush]

Our current recommendation follows the ACMG set f clinical significance terms (P, LP, B, LB, VUS). But this approach conflates the significance classification with the asserting agent's confidence that it is true.

These pieces of information can be separated, to increase flexibility and computability and interoperability of the data. But in doing so we would move away from the established terms used by the community.

The proposal here is to limit the clinical significance terms recommended for the predicate slot of a Variant Pathogenicity classification to:

• Pathogenic
• Benign
• Uncertain Significance

And use the confidence_level field on the Statement to capture confidence info reflecting how sure the agent is:

• likely
• definitive

[larrybabb]

For the Variant Pathogenicity Classification statement there is an inherent quality to each of the three terms that have importance as well. Pathogenic is clinically significant, Benign is not clinically signficant, and Uncertain Significance is what it is.

Is the Confidence term in the Assertion a claim about the confidence in the findings/evidence for supporting or refuting the assertion? Or is the Confidence term in the Assertion a qualifier for the underlying statement's predicate?

In other words, could someone be highly confident that something is likely based on the findings/evidence that supports a likely statement? Confidence seems like an arbitrary scale (i.e. 0-10 where 0 is no confidence and 10 is the highest confidence possible). If this is confidence in a statement, then the statement can be literally anything... i think. Including statements that show lack of certainty.

Let's put confidence, direction and negation on the table along with any other qualifiers that impact the outcome of an assertion to make sure we understand precisely what each element is meant to do and verify that we don't have a conflation of concerns.

[mbrush]

On 3-31-21 Call we decided to defer this work / decision to v1. Some folks indicated this change could be very disruptive. We need to better understand nuances around confidence assessments of VPI assertions in this community of practice, before we are ready to make a proposal here as well. Dmitriy shared this document about quantitative scoring of ACMG criteria / evidence, which may be relevant: https://onlinelibrary.wiley.com/doi/10.1002/humu.24088

[ahwagner]

Just adding a comment here I found interesting. In discussing with Dr. Marilyn Li at CGC, the notion of "oncogenicity" is not determined "clinically significant" by itself, but only in conjunction with statements of clinical significance as defined by the AMP/ASCO/CAP somatic classification standard.

So in thinking about "uncertain" as a classification label in future conversations, we may wish to separate the notion of "clinical significance" from "classification certainty". This is certainly confusing as the Horak et al. paper uses "VUS" as an oncogenicity classification inherited from the ACMG guidelines, but in the ACMG guidelines VUS refers to a variant of uncertain clinical significance, which wouldn't make sense from an oncogenicity perspective if Dr. Li's perspective is reflective of the field (though I agree with Dr. Li on this point).

[larrybabb]

does the variation-condition association classification have clinical significance or not? (or are you uncertain), that is what needs to be captured. I would think this is just as relevant whether it's somatic cancer or germline disease statements. I'm not sure I understand your comment above that oncogenicity is more about "classificaiton certainty".

classification certainty conveys a notion of "how certain are we about our classification?" That seems like a strength level. Being "uncertain" about the oncogenicity classification is fine and may have nothing to do with VUS, but that just means "I'm not making a classification call because I don't have enough evidence. If the oncogenicity folks do anything with the idea of "conflicting evidence" or "evidence that is not quite strong enough to say its oncogenic/likely oncogenic vs benign/likely benign, then uncertain significance is the same thing for oncogenicity classifications as pathogenicity ones.

If they are different then I would say that the pathogenicity folks would be misusing "VUS" as "Uncertain Significance".

I'm out of my league here, but I'd feel a heck of a lot better if someone like Heidi spoke with you (@ahwagner ) and Dr Li, to make sure we are talking apples-and-oranges (chalks and cheeses for our friends in England) or apples-and-apples (chalks and chalks ? ).

[larrybabb]

BTW - there's a forthcoming ACMG guideline revision of the Pathogenicity work that will provide a quantifiable score for the evidence which will map to a scale that will help discern when the "uncertain-path" or "uncertain-benign" thresholds are crossed. I don't think we should contrive a numeric confidence score because we have no policy/method or paper that we can be sure that everyone would conform to. All we have today is a way to breakdown the 2 levels of "confidence" that can be mapped to "likely" and "definitive" for the pathogenicity terms. This 2-level confidence differences between the significant (path) and non-significant (benign) classifications are the only "confidence levels" that are needed for the time being. It almost makes sense to not try to model this until we have more evidence of a need for the "confidence level" and just simply put the precise "qualified" terms into the classification. But I digress. The 2-level terms "definitive" and "likely" will be good enough for the time being IMO.

[ahwagner]

I think we're on the same page @larrybabb and this won't have any bearing on the way we have discussed propositions & statements to date, specifically around germline conditions. This is primarily a concern about the ClinGen Somatic / CGC / VICC oncogenicity SOP and the use of "VUS" as a classification category. We don't need to model assertions of that type before plenary, it's a pretty new specification, so we will have plenty time to discuss and build consensus across the broader community before implementation.