LocalIds in annotations missing in ELM nodes

[birick1]

While investigating coverage calculation and highlighting post ELM translator v3.6.0, we identified at least one case where there are localIds in the annotation but not in the ELM expression. This leads to a problem in correctly highlighting statements for coverage.

This item is raised in JIRA at https://oncprojectracking.healthit.gov/support/browse/CQLIT-432

For example, we are finding this to be an issue with CQL Query relationships (i.e. with). In the ELM json for AHAOverall-2.6.000.json (in MAT6725-v0.0.000-FHIR4_Bad), the localId 309 that we need for highlighting of the HeartFailure alias is only present in the ELM annotation but not in the ELM expressions. We were able to replicate this error with a smaller CQL file that contained a with statement.

Attaching referenced file, and more info can be found in the JIRA ticket: MAT6725-v0.0.000-FHIR4_Bad.zip

[JPercival]

The generated ELM for that with the latest CQL release includes a localId for that HeartFailure alias:

<relationship localId="81" locator="41:5-43:53" alias="HeartFailure" xsi:type="With">
               <resultTypeSpecifier xsi:type="ListTypeSpecifier">
                  <elementType name="fhir:Condition" xsi:type="NamedTypeSpecifier"/>
               </resultTypeSpecifier>
               <expression localId="41" locator="41:10-41:37" dataType="fhir:Condition" templateId="http://hl7.org/fhir/us/qicore/StructureDefinition/qicore-condition" codeProperty="code" codeComparator="in" xsi:type="Retrieve">
                  <resultTypeSpecifier xsi:type="ListTypeSpecifier">
                     <elementType name="fhir:Condition" xsi:type="NamedTypeSpecifier"/>
                  </resultTypeSpecifier>
                  <codes locator="41:22-41:36" resultTypeName="t:ValueSet" name="Heart Failure" preserve="true" xsi:type="ValueSetRef"/>
               </expression>
               <suchThat localId="80" locator="42:17-43:53" resultTypeName="t:Boolean" xsi:type="And">
                  <signature name="t:Boolean" xsi:type="NamedTypeSpecifier"/>
                  <signature name="t:Boolean" xsi:type="NamedTypeSpecifier"/>
                  <operand localId="47" locator="42:17-42:85" resultTypeName="t:Boolean" xsi:type="Overlaps">
                     <signature xsi:type="IntervalTypeSpecifier">
                        <pointType name="t:DateTime" xsi:type="NamedTypeSpecifier"/>
                     </signature>
                     <signature xsi:type="IntervalTypeSpecifier">
                        <pointType name="t:DateTime" xsi:type="NamedTypeSpecifier"/>
                     </signature>
                     <operand localId="44" locator="42:17-42:49" name="prevalenceInterval" libraryName="QICoreCommon" xsi:type="FunctionRef">
                        <resultTypeSpecifier xsi:type="IntervalTypeSpecifier">
                           <pointType name="t:DateTime" xsi:type="NamedTypeSpecifier"/>
                        </resultTypeSpecifier>
                        <signature name="fhir:Condition" xsi:type="NamedTypeSpecifier"/>
                        <operand localId="43" locator="42:17-42:28" resultTypeName="fhir:Condition" name="HeartFailure" xsi:type="AliasRef"/>
                     </operand>
                     <operand localId="46" locator="42:60-42:85" name="ToInterval" libraryName="FHIRHelpers" xsi:type="FunctionRef">
                        <resultTypeSpecifier xsi:type="IntervalTypeSpecifier">
                           <pointType name="t:DateTime" xsi:type="NamedTypeSpecifier"/>
                        </resultTypeSpecifier>
                        <operand path="period" scope="QualifyingEncounter" xsi:type="Property"/>
                     </operand>
                  </operand>
                  <operand localId="79" locator="43:13-43:53" resultTypeName="t:Boolean" name="isConfirmedActiveDiagnosis" xsi:type="FunctionRef">
                     <signature name="fhir:Condition" xsi:type="NamedTypeSpecifier"/>
                     <operand localId="48" locator="43:13-43:24" resultTypeName="fhir:Condition" name="HeartFailure" xsi:type="AliasRef"/>
                  </operand>
               </suchThat>
            </relationship>

Are you seeing this only on JSON?

[birick1]

Yes - we are only looking at the JSON.

[JPercival]

I've been unable to replicate the issue but perhaps I'm misunderstanding issue. Here's a simplified example CQL definition and a subset of the associated JSON being produced by the CQL compiler. Could you point out to me where you're expecting localId to appear?

define "Encounter with Condition":
  [Encounter] E
    with [Condition] C
      such that C.onset overlaps E.period

    "statements" : {
      "type" : "Library$Statements",
      "def" : [ {
        "type" : "ExpressionDef",
        "expression" : {
          "type" : "Query",
          "source" : [ {
            "type" : "AliasedQuerySource",
            "expression" : {
              "type" : "Retrieve",
              "localId" : "211",
              "locator" : "8:3-8:13",
              "dataType" : "{http://hl7.org/fhir}Encounter",
              "templateId" : "http://hl7.org/fhir/StructureDefinition/Encounter"
            },
            "localId" : "210",
            "locator" : "8:3-8:15",
            "alias" : "E"
          } ],
          "relationship" : [ {
            "type" : "With",
            "expression" : {
              "type" : "Retrieve",
              "localId" : "213",
              "locator" : "9:10-9:20",
              "dataType" : "{http://hl7.org/fhir}Condition",
              "templateId" : "http://hl7.org/fhir/StructureDefinition/Condition"
            },
            "suchThat" : {
              "type" : "Overlaps",
              "signature" : [ {
                "type" : "IntervalTypeSpecifier",
                "pointType" : {
                  "type" : "NamedTypeSpecifier",
                  "localId" : "225",
                  "name" : "{urn:hl7-org:elm-types:r1}DateTime"
                },
                "localId" : "224"
              }, {
                "type" : "IntervalTypeSpecifier",
                "pointType" : {
                  "type" : "NamedTypeSpecifier",
                  "localId" : "227",
                  "name" : "{urn:hl7-org:elm-types:r1}DateTime"
                },
                "localId" : "226"
              } ],
              "operand" : [ {
                "type" : "FunctionRef",
                "signature" : [ {
                  "type" : "NamedTypeSpecifier",
                  "localId" : "221",
                  "name" : "{http://hl7.org/fhir}Period"
                } ],
                "operand" : [ {
                  "type" : "As",
                  "operand" : {
                    "type" : "Property",
                    "localId" : "215",
                    "locator" : "10:17-10:23",
                    "path" : "onset",
                    "scope" : "C"
                  },
                  "localId" : "219",
                  "asType" : "{http://hl7.org/fhir}Period"
                } ],
                "localId" : "220",
                "name" : "ToInterval",
                "libraryName" : "FHIRHelpers"
              }, {
                "type" : "FunctionRef",
                "signature" : [ {
                  "type" : "NamedTypeSpecifier",
                  "localId" : "223",
                  "name" : "{http://hl7.org/fhir}Period"
                } ],
                "operand" : [ {
                  "type" : "Property",
                  "localId" : "217",
                  "locator" : "10:34-10:41",
                  "path" : "period",
                  "scope" : "E"
                } ],
                "localId" : "222",
                "name" : "ToInterval",
                "libraryName" : "FHIRHelpers"
              } ],
              "localId" : "218",
              "locator" : "10:17-10:41"
            },
            "localId" : "228",
            "locator" : "9:5-10:41",
            "alias" : "C"
          } ],
          "localId" : "229",
          "locator" : "8:3-10:41"
        },
        "annotation" : [ {
          "type" : "Annotation",
          "s" : {
            "s" : [ {
              "name" : "{urn:hl7-org:cql-annotations:r1}s",
              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
              "value" : {
                "s" : [ "// https://github.com/cqframework/clinical_quality_language/issues/1338\n", "define ", "\"Encounter with Condition\"", ":\n  " ]
              },
              "nil" : false,
              "globalScope" : true,
              "typeSubstituted" : false
            }, {
              "name" : "{urn:hl7-org:cql-annotations:r1}s",
              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
              "value" : {
                "s" : [ {
                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                  "value" : {
                    "s" : [ {
                      "name" : "{urn:hl7-org:cql-annotations:r1}s",
                      "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                      "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                      "value" : {
                        "s" : [ {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "[", "Encounter", "]" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "211"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            } ],
                            "r" : "211"
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        }, {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ " ", "E" ]
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        } ],
                        "r" : "210"
                      },
                      "nil" : false,
                      "globalScope" : true,
                      "typeSubstituted" : false
                    } ]
                  },
                  "nil" : false,
                  "globalScope" : true,
                  "typeSubstituted" : false
                }, {
                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                  "value" : {
                    "s" : [ "\n    " ]
                  },
                  "nil" : false,
                  "globalScope" : true,
                  "typeSubstituted" : false
                }, {
                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                  "value" : {
                    "s" : [ {
                      "name" : "{urn:hl7-org:cql-annotations:r1}s",
                      "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                      "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                      "value" : {
                        "s" : [ "with " ]
                      },
                      "nil" : false,
                      "globalScope" : true,
                      "typeSubstituted" : false
                    }, {
                      "name" : "{urn:hl7-org:cql-annotations:r1}s",
                      "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                      "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                      "value" : {
                        "s" : [ {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "[", "Condition", "]" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "213"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            } ],
                            "r" : "213"
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        }, {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ " ", "C" ]
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        } ],
                        "r" : "212"
                      },
                      "nil" : false,
                      "globalScope" : true,
                      "typeSubstituted" : false
                    }, {
                      "name" : "{urn:hl7-org:cql-annotations:r1}s",
                      "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                      "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                      "value" : {
                        "s" : [ "\n      such that " ]
                      },
                      "nil" : false,
                      "globalScope" : true,
                      "typeSubstituted" : false
                    }, {
                      "name" : "{urn:hl7-org:cql-annotations:r1}s",
                      "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                      "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                      "value" : {
                        "s" : [ {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "C" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "214"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            }, {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ "." ]
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            }, {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "onset" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "215"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            } ],
                            "r" : "215"
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        }, {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ " ", "overlaps", " " ],
                            "r" : "218"
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        }, {
                          "name" : "{urn:hl7-org:cql-annotations:r1}s",
                          "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                          "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                          "value" : {
                            "s" : [ {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "E" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "216"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            }, {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ "." ]
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            }, {
                              "name" : "{urn:hl7-org:cql-annotations:r1}s",
                              "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                              "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                              "value" : {
                                "s" : [ {
                                  "name" : "{urn:hl7-org:cql-annotations:r1}s",
                                  "declaredType" : "org.hl7.cql_annotations.r1.Narrative",
                                  "scope" : "jakarta.xml.bind.JAXBElement$GlobalScope",
                                  "value" : {
                                    "s" : [ "period" ]
                                  },
                                  "nil" : false,
                                  "globalScope" : true,
                                  "typeSubstituted" : false
                                } ],
                                "r" : "217"
                              },
                              "nil" : false,
                              "globalScope" : true,
                              "typeSubstituted" : false
                            } ],
                            "r" : "217"
                          },
                          "nil" : false,
                          "globalScope" : true,
                          "typeSubstituted" : false
                        } ],
                        "r" : "218"
                      },
                      "nil" : false,
                      "globalScope" : true,
                      "typeSubstituted" : false
                    } ],
                    "r" : "228"
                  },
                  "nil" : false,
                  "globalScope" : true,
                  "typeSubstituted" : false
                } ],
                "r" : "229"
              },
              "nil" : false,
              "globalScope" : true,
              "typeSubstituted" : false
            } ],
            "r" : "209"
          }
        } ],
        "localId" : "209",
        "locator" : "7:1-10:41",
        "name" : "Encounter with Condition",
        "context" : "Unfiltered",
        "accessLevel" : "Public"
      } ]
    }

[birick1]

Hi @JPercival - thank you for the example.

In the Bad.zip file, localId 309 that we need for highlighting of the HeartFailure alias is only present in the ELM annotation but not in the ELM expressions. In addition, when this occurs a duplicate localId is placed within the annotation and used once in the ELM expressions - in the zip file case it is id 312 (see image)

Coming to the example you've provided, I think the issue is being replicated with localId 212 (only in the annotation) and localId 213 (occurs twice in the annotation).

What's throwing me off and giving me some uncertainty is the structure of the json you've provided seems different. For example, in your example, there is "value": {"s": ["with "]}, whereas in Bad.zip it is "s": [{"value": ["with "]}]

[JPercival]

For 213 it looks as though narrative chunks are not being collapsed correctly. I'll look at that.

For 212 I suspect the issue may be similar. The complete JSON has localIds assigned, but associated with a completely different ELM elements.

[JPercival]

Hi @birick1, could you give the branch here a shot? I think we've got it resolved: https://github.com/cqframework/clinical_quality_language/pull/1356

[JPercival]

1356 merged to master