Implementation of $apply API

fredhersch commented 3 years ago

User story: As an implementer I can use the $apply operation to generate a CarePlan for a patient given a PlanDefinition

Background CQL (Clinical Quality Language) is the HL7 standard for providing logic execution for decision support and clinical quality measures.

Our initial goal with the CQL Library will be to demonstrate over time the ability to:

Generate a MeasureReport via $measure-evaluate operation
Load and execute a custom CQL library (Library resource) for decision logic
Transform a PlanDefinition to a CarePlan using $apply operation

jingtang10 commented 3 years ago

dependencies:

maimoonak commented 2 years ago

Few Observations during code exploration

CRUD in CQL, does fhir spec allow it or not?
CRUD in Measure, does fhir spec allowsit or not?
Library Processor L213 -> if (!url.contains("/Library/"))
Library name RuleFilter
ELM+JSON LIbrary works with CQL. XML throws Serializer exception
Translating Library matcher translatorOptionsMatch needs replacement with matcher method
The CQL Builder with default constructor does not initialize properly with library loader hence missing libraries on runtime
libraryCache needs to be carefully implemented, currently alot of initialization happens with in classes privately
The FHIRDal i.e. CQL would be loading data on demand. This restricts us running CQL before saving entities (For example we just want to test questionnaire, extraction, and CQL without persisting entities)
ExpressionEvaluator runs expression as library but in resources no content / library has been provided (removing expressions make testcase pass) libraryProcessor.evaluate(new VersionedIdentifier().withId("expression").withVersion("1.0.0"), subject,

Plz check commit here. These changes were required to run the library loaders. The test does not pass though due to expression not added as library (Not sure if I am missing anything here)

@f-odhiambo @pld @jingtang10 @dubdabasoduba

f-odhiambo commented 2 years ago

Seem like we have the following

Input

val cp = fhirOperator.generateCarePlan(
      planDefinitionId = “plandefinition-RuleFilters-1.0.0”,
      patientId = “Reportable”,
      encounterId = “reportable-encounter”
    )

and gettging the following output based on @maimoonak updates

Output {"resourceType":"CarePlan","contained":[{"resourceType":"RequestGroup","id":"3ddb8524-345a-4477-9589-4144b71ff88c","status":"draft","intent":"proposal"}],"instantiatesCanonical":["plandefinition-RuleFilters-1.0.0"],"status":"draft","subject":{"reference":"Reportable"},"encounter":{"reference":"reportable-encounter"},"activity":[{"reference":{"reference":"#3ddb8524-345a-4477-9589-4144b71ff88c"}}]}

Are you getting the same @jingtang10?

CC @pld @mberg

pld commented 2 years ago

Can we re-open this issue? I don't believe the above issues raised by @maimoonak and @f-odhiambo were addressed and tested. But maybe I am missing those tests, can you link to them?

jingtang10 commented 2 years ago

Thanks @pld for raising this. In the call earlier today we discussed this and @maimoonak will share the changes she has made and raise a draft pr against this issue.

also would love to get input from jp and vitor (sent them invite to the repo).

Tarun-Bhardwaj commented 2 years ago

@f-odhiambo , though the associated PR #1195 is merged, what would it need to close this issue?

f-odhiambo commented 2 years ago

I think we can track the $apply issues based on @vitorpamplona updates. I am ok closing this issue. Will update and coordinate with @maimoonak on where we can push the subsequent updates CC @Tarun-Bhardwaj

jingtang10 commented 2 years ago

Few Observations during code exploration

CRUD in CQL, does fhir spec allow it or not?

CRUD in Measure, does fhir spec allowsit or not?

Library Processor L213 -> if (!url.contains("/Library/"))

Library name RuleFilter

ELM+JSON LIbrary works with CQL. XML throws Serializer exception

Translating Library matcher translatorOptionsMatch needs replacement with matcher method

The CQL Builder with default constructor does not initialize properly with library loader hence missing libraries on runtime

libraryCache needs to be carefully implemented, currently alot of initialization happens with in classes privately

The FHIRDal i.e. CQL would be loading data on demand. This restricts us running CQL before saving entities (For example we just want to test questionnaire, extraction, and CQL without persisting entities)

ExpressionEvaluator runs expression as library but in resources no content / library has been provided (removing expressions make testcase pass) libraryProcessor.evaluate(new VersionedIdentifier().withId("expression").withVersion("1.0.0"), subject,

Plz check commit here. These changes were required to run the library loaders. The test does not pass though due to expression not added as library (Not sure if I am missing anything here)

@f-odhiambo @pld @jingtang10 @dubdabasoduba

Let's close this issue once we address everything in @maimoonak 's commit in her comment above. @maimoonak please close this issue or raise a PR if necessary.

maimoonak commented 2 years ago

Current code has $apply unit test ignored hence we never knew that it is not working. I have removed the Ignore and now test fails. The PR https://github.com/google/android-fhir/pull/1471 resolve some of the issues we need to address to run the test. I could not find a solution to last one. Looks like there is a progress from previous state but it still does not work. The step by step exception vs solution are below

the RuleFilters-1.0.0-bundle.json file might be collapsed into PR so plz expand to see the changes

1- Issue in cql library where it expects that the URL would be from server, however, by fhir definition it can be Canonical Local URL. Change PlanDefinition prefix / to -> "library": [ "Library/library-RuleFilters-1.0.0" ],

Invalid resource type for determining library version identifier: Library
java.lang.IllegalArgumentException: Invalid resource type for determining library version identifier: Library
    at org.opencds.cqf.cql.evaluator.library.LibraryProcessor.getVersionedIdentifer(LibraryProcessor.java:214)
    at org.opencds.cqf.cql.evaluator.library.LibraryProcessor.evaluate(LibraryProcessor.java:93)

2- The cql loader internally work (compare and search) on library name so change the Library to correct reference i.e. "name": "RuleFilters" -> "name": "library-RuleFilters-1.0.0"

java.lang.NullPointerException
    at com.google.android.fhir.workflow.FhirEngineLibraryContentProvider.getLibrary(FhirEngineLibraryContentProvider.kt:30)
    at org.opencds.cqf.cql.evaluator.cql2elm.content.fhir.BaseFhirLibraryContentProvider.getLibraryContent(BaseFhirLibraryContentProvider.java:35)
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.getLibraryContent(TranslatingLibraryLoader.java:112)

3- The expression in condition matchers throw exception -> Remove null check on FhirEngineLibraryContentProvider as internally it creates library and adds it to InMemoryLibraryLoader and let cql engine search into all library providers

java.lang.NullPointerException
    at com.google.android.fhir.workflow.FhirEngineLibraryContentProvider.getLibrary(FhirEngineLibraryContentProvider.kt:30)
    at org.opencds.cqf.cql.evaluator.cql2elm.content.fhir.BaseFhirLibraryContentProvider.getLibraryContent(BaseFhirLibraryContentProvider.java:35)
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.getLibraryContent(TranslatingLibraryLoader.java:112)

4- For expression as library org.opencds.cqf.cql.evaluator.expression.ExpressionEvaluator#getFhirVersion hardcodes fhir version 4.0.1 hence throw exception -> library added to Bundle but also needed change in FhirEngineLibraryContentProvider.libs to make key a Pair of name,version to pick correct library when needed

Translation of library expression failed with the following message: Library FHIRHelpers was included as version 4.0.1, but version 4.0.0 of the library was found.
org.opencds.cqf.cql.engine.exception.CqlException: Translation of library expression failed with the following message: Library FHIRHelpers was included as version 4.0.1, but version 4.0.0 of the library was found.
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:134)
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)
    at org.opencds.cqf.cql.evaluator.engine.execution.CacheAwareLibraryLoaderDecorator.load(CacheAwareLibraryLoaderDecorator.java:49)

5- FhirEngineLibraryContentProvider updated to check for name and version if given, otherwise if version is null just return matching library by name

Unable translate library library-RuleFilters-1.0.0
org.opencds.cqf.cql.engine.exception.CqlException: Unable translate library library-RuleFilters-1.0.0
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:128)
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)

6- Unresolved

Translation of library expression failed with the following message: Could not resolve identifier encounter in the current library.
org.opencds.cqf.cql.engine.exception.CqlException: Translation of library expression failed with the following message: Could not resolve identifier encounter in the current library.
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:134)
    at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)
    at org.opencds.cqf.cql.evaluator.engine.execution.CacheAwareLibraryLoaderDecorator.load(CacheAwareLibraryLoaderDecorator.java:49)

@jingtang10 @pld @f-odhiambo @dubdabasoduba

jingtang10 commented 2 years ago

@vitorpamplona @brynrhodes can you help here please?

vitorpamplona commented 2 years ago

Fixed on https://github.com/google/android-fhir/pull/1603

google / android-fhir

Implementation of $apply API #656