projecttacoma / fqm-execution

fqm-execution is a library that allows users to calculate FHIR-based electronic Clinical Quality Measures (eCQMs) and retrieve the results in a variety of formats
https://projecttacoma.github.io/fqm-execution/
Apache License 2.0
18 stars 6 forks source link

Coverage Drops After Incorporating Translator v3.14.0 #313

Open jkotanchik-SB opened 1 week ago

jkotanchik-SB commented 1 week ago

Summary

After updating to the 3.14.0 cql-to-elm Translator, our QA team identified reduced Coverage values on some hand-picked measures:

# Measure 3.14.0 3.3.2
1 CMS135FHIR 99% 100%
2 CMS144FHIR 96% 100%
3 CMS145FHIR 99% 100%
4 CMS159FHIR 99% 100%
5 CMS645FHIR 94% 99%
6 CMS646FHIR 95% 99%
7 CMS771FHIR 94% 100%

External Tracking Ticket

N/A

Expected Behavior

Coverage matches when using older and current cql-to-elm version.

Version or Commit

v1.5.0

Inputs (e.g. Measure Bundle, Patient Bundle, CQL Library)

MeasureBundles.zip

Patient Bundles:

  1. CMS135FHIR-v0.2.000-FHIR4-TestCases.zip
  2. CMS144FHIR-v1.4.000-FHIR4-TestCases.zip
  3. CMS145FHIR-v0.2.000-FHIR4-TestCases.zip
  4. CMS159FHIR-v0.2.000-FHIR4-TestCases.zip
  5. CMS645FHIR-v1.4.000-FHIR4-TestCases.zip
  6. CMS646FHIR-v1.4.000-FHIR4-TestCases.zip
  7. CMS771FHIR-v1.4.000-FHIR4-TestCases.zip

Relevant Calculation Options (e.g. Measurement Period, meta.profile Validation)

{
    includeClauseResults: false,
    trustMetaProfile: true,
    buildStatementLevelHTML: true,
    measurementPeriodStart: 1/1/2025,
    measurementPeriodEnd: 12/31/2025,
}

cql-to-elm Version Used for Measure Logic Translation (if known)

v3.14.0

Any Additional Info

Similar to #285, with the updated cql-to-elm translator, the highlighting is showing aliases uncovered.

elsaperelli commented 1 day ago

Hi @jkotanchik-SB ! Could you provide us with the previous measure bundles that used the 3.3.2 version of the translator? Thank you!

jkotanchik-SB commented 11 hours ago

Re-built the Measure bundles using the 3.3.2 Translator. There might be some minor differences due to having to pull the measure content from PROD into my local to complete the rebuild. (For instance, random whitespace in the CQL so that I could enable the save changes button to complete the rebuild in MADiE.)

MeasureBundles-3_3_2.zip