Test Case Coverage Is not 100% as expected

[RohitKandimalla]

Summary

The MADiE team received a helpdesk ticket, where the user is complaining that a few clauses in CQL are not highlighted in Coverage when they are expected to be. Can you guys take a look at its measure and let us know the following questions?

1. Are fluent functions expected to be highlighted?
2. SDE Payer not highlighting as tested even though the payer is not qualified in the logic.
3. 4 different aliases are not showing as covered.

All questions that the user has is related to highlighting and I attached the actual ticket number for reference, The ONC ticket has additional information regarding the expectations vs actual coverage. (Documentation of lack of test case coverage.docx , and CMS334 Test Case Example.docx) In addition, I am attaching measureBundle, patientBundle, and valueSetExpansions that MADiE used for calculations.

Note: The patient bundle attached doesn't contain all the patients, rather it has only 1, where fluent functions are not highlighted. Please let us if we need to include all test cases, There are a lot, but I think I can work on it.

External Tracking Ticket

BONNIEMAT-1479

Expected Behavior

Expected to have full coverage, that includes SDE Payor, Fluent functions

Version or Commit

Inputs (e.g. Measure Bundle, Patient Bundle, CQL Library)

ValueSetExpansions.txt PatientBundle.txt measureBundle.txt

Relevant Calculation Options (e.g. Measurement Period, meta.profile Validation)

cql-to-elm Version Used for Measure Logic Translation (if known)

Any Additional Info

[mgramigna]

Hi @RohitKandimalla, I'll take these one a time:

1. See Peter's response on the ticket. Both regular CQL functions and fluent functions are explicitly excluded from both the clause coverage percentage calculation and the HTML highlighting. They are not expected to be highlighted, and are not contributing to the coverage percentage number at all
2. For the SDE Payer issue, this looks like an issue in the patient data to me. At least with the patient data that is posted to this issue, it does not have a .type property, and the CQL specifically looks for a Coverage resource with a .type in the "Payer" valueset:

define "SDE Payer":
  [Coverage: type in "Payer"] Payer
    return {
      code: Payer.type,
      period: Payer.period
    }

I added a .type with a code from the Payer valueset to the Coverage resource in the patient bundle attached to this ticket, and the coverage/logic is highlighting in the HTML as expected. I know this is only one patient in a potential set of many that were part of this initial issue in MADiE, but if none of those patients had the proper .type attribute, that part of CQL will not highlight.

3. I was able to replicate the issue with the aliases, however, it looks like the lack of coverage highlighting on the aliases also does not decrease the clause coverage percentage:

Here I have an example where you see the alias not highlighting properly, but the coverage percentage value is 100% still. This seems to be an issue with the ELM annotations/alias lookup that we do in fqm-execution. We can look into that issue to try and make sure that all uses of the alias highlight appropriately, but looks like that won't change the coverage percentage at all.

---

To summarize, I would say make sure the proper .type code exists on at least one of the Coverage resources on a test case. That should lead to the "SDE Payer" expression properly being covered. If at that point, the coverage percentage is still not as expected, we can try to isolate any other potential issues with the coverage and what is causing the delta in percentage, since it is not the aliases.

[RohitKandimalla]

Thank you @mgramigna !! We have provided your response to the user. I will give a couple of days to see if the user is having any other issues, if not I will close this issue.

[p9g]

for discussion