Coverage for Negation Aliases

jkotanchik-SB commented 9 months ago

Summary

Another round of Measures experiencing issues reaching 100% coverage. It may be an extension of #285 and I move these materials to that issue if it makes sense.

The potential difference being the problematic aliases deal with negation, if that's a difference at all.

Impacted Measures:

CMS 2 Preventive Care and Screening: Screening for Depression and Follow-Up PlanFHIR
CMS 22 Preventive Care and Screening: Screening for High Blood Pressure and Follow-Up DocumentedFHIR
CMS69 Preventive Care and Screening: Body Mass Index (BMI) Screening and Follow-Up PlanFHIR
CMS347 Statin Therapy for the Prevention and Treatment of Cardiovascular DiseaseFHIR

External Tracking Ticket

https://oncprojectracking.healthit.gov/support/browse/BONNIEMAT-1631

Expected Behavior

100% coverage

Version or Commit

v1.3.1

Inputs (e.g. Measure Bundle, Patient Bundle, CQL Library)

Measure Bundles:

MADiE Patient Exports:

Relevant Calculation Options (e.g. Measurement Period, meta.profile Validation)

{
    includeClauseResults: false,
    trustMetaProfile: true,
    buildStatementLevelHTML: true,
    measurementPeriodStart: 1/1/2025,
    measurementPeriodEnd: 12/31/2025,
}

cql-to-elm Version Used for Measure Logic Translation (if known)

v2.11.0

Any Additional Info

hossenlopp commented 9 months ago

Hi @jkotanchik-SB,

The aliases, unions and library references that visually show as uncovered do not affect coverage percentage calculations. These are caused by quirks in the ELM annotations.

We have just released v1.3.2 which fixes an issue with visual coverage and should now properly show locations in the logic that are uncovered in many cases. This update should make it clear where those last few uncovered clauses are and aid in creating additional test cases.

Please test with this release and open a new issue if there are additional problems with coverage or you are unable to get to 100% after satisfying the clauses that will now properly show as uncovered.

sb-cecilialiu commented 8 months ago

@hossenlopp Good afternoon! We tested the newly released v1.3.2 but it seems the issues are still not resolved. I'm attaching the measures and test cases for the testing of the issue for your reference, hopefully this issue can be re-opened soon and looked into. Thanks! CMS2FHIR-v0.2.000-FHIR4-TestCases.zip CMS2FHIR-v0.2.000-FHIR4.zip CMS22FHIR-v0.2.000-FHIR4-TestCases.zip CMS22FHIR-v0.2.000-FHIR4.zip CMS69FHIR-v0.2.000-FHIR4-TestCases.zip CMS69FHIR-v0.2.000-FHIR4.zip CMS347FHIR-v0.2.000-FHIR4-TestCases.zip CMS347FHIR-v0.2.000-FHIR4.zip

hossenlopp commented 8 months ago

Hi @sb-cecilialiu,

Thanks for the updated packages.

CMS2 and CMS22 do not have issues with coverage percentage calculation. The aliases that are not highlighting do not affect calculation as they are a quirk in the ELM annotations. This is not something we are attempting to resolve at the moment. The given test cases for these measures do not reach 100% coverage because there are parts of the logic that are not covered by the test cases. The highlighting in v1.3.2 does accurately show where these testing gaps are.

CMS69 and CMS347 will require additional investigation by our team. We will prioritize this in our upcoming sprint.

-Chris

elsaperelli commented 7 months ago

The issue with coverage percentage calculation for the !~ (Not Equivalent) clause in CMS69 was fixed in #291 and released in fqm-execution 1.3.3. The coverage calculation percentage was not reaching 100% due to how the cql-to-elm translator currently translates this CQL. This also accounts for potential translator changes for != (Not Equal). The attached document goes into more detail on this issue.

fqm-execution Not Equivalent Handling.pdf

elsaperelli commented 5 months ago

Hi @sb-cecilialiu @jkotanchik-SB

Our team has looked into CMS347 and was able to get 100% clause coverage for each of the four population groups with additional test cases. The clause coverage highlighting reflects the 100% clause coverage, with the exception of aliases, unions and library references that visually show as uncovered but do not affect coverage percentage calculations. As previously mentioned, these are caused by quirks in the ELM annotations.

Thank you!

sb-cecilialiu commented 5 months ago

@elsaperelli Thank you so much!

projecttacoma / fqm-execution