Inferno AU Core tests for Suppressed Data, and Missing Data

[projkov]

1. Check how the validator works with suppressed data;
2. Create a synthetic data set that demonstrates suppressed and missing data (success and failed cases);
3. Report to the team and discuss a particular implementation plan.

[projkov]

Notes

As I can see, the validator works okay with missing data. A good example is the birth date of the patient which can be filled as "unknown" by using _birthDate element.

[projkov]

An example of the Patient resource with missing name, gender, date of birth

{
  "resourceType": "Patient",
  "id": "banks-mia-leanne",
  "meta": {
    "profile": [
      "http://hl7.org.au/fhir/core/StructureDefinition/au-core-patient"
    ]
  },
  "name": [
    {
      "_family": {
        "extension": [
          {
            "url": "http://hl7.org/fhir/StructureDefinition/data-absent-reason",
            "valueCode": "unknown"
          }
        ]
      }
    }
  ],
  "gender": "unknown",
  "_birthDate": {
    "extension": [
      {
        "url": "http://hl7.org/fhir/StructureDefinition/data-absent-reason",
        "valueCode": "unknown"
      }
    ]
  },
  "identifier": [
    {
      "extension": [
        {
          "url": "http://hl7.org.au/fhir/StructureDefinition/ihi-status",
          "valueCoding": {
            "system": "https://healthterminologies.gov.au/fhir/CodeSystem/ihi-status-1",
            "code": "active"
          }
        },
        {
          "url": "http://hl7.org.au/fhir/StructureDefinition/ihi-record-status",
          "valueCoding": {
            "system": "https://healthterminologies.gov.au/fhir/CodeSystem/ihi-record-status-1",
            "code": "verified",
            "display": "verified"
          }
        }
      ],
      "type": {
        "coding": [
          {
            "system": "http://terminology.hl7.org/CodeSystem/v2-0203",
            "code": "NI"
          }
        ],
        "text": "IHI"
      },
      "system": "http://ns.electronichealth.net.au/id/hi/ihi/1.0",
      "value": "8003608333647261"
    }
  ]
}

[projkov]

The mechanism and logic for handling suppressed and missing data are correct.

We understand that the official HL7 validator can be used to check resources with missing data.

The question is: how should the tests be implemented?

[projkov]

Hello, everyone @StephThaoOng @heathfrankel @ir4y

The test for US Core works as follows: For each validation test on any resource, Inferno will validate the resource, followed by a check for the Data Absent Reason (DAR).

This check consists of two steps: 1. Check each element of the resource for the DAR code: This involves iterating through each element of the resource. If an element is identified as an FHIR Coding element, it will be sent to a method that checks whether the coding.code is equal to 'unknown' and the coding.system is equal to DAR_CODE_SYSTEM. If both conditions are met, the test sets a global variable related to code to TRUE, indicating that the DAR code test is successful. 2. Check the entire resource for the presence of the DAR extension: This step involves checking if the DAR extension URL is present in the resource. If found, the test sets the global variable related to an extension to TRUE, indicating that the DAR extension test is successful.

Pros:

1. It's simple and already implemented.

Cons:

1. To pass this test, only a single resource with DAR is required.

My suggestion: We can implement an optional test with the following algorithm:

1. Retrieve all resources from previous tests.
2. Group all resources by resourceType.
3. Attempt to check for the DAR extension or DAR code for each resource.
4. Log the results to the test output in a format similar to the following:

   AU Core Patient resources: DAR code exists, DAR extension exists.
   AU Core Blood Pressure resources: DAR code exists, DAR extension does not exist.

This approach would be more informative than the current method, which checks only a single code and extension.

[StephThaoOng]

@projkov Thanks for summarising how the US Core test works. Your suggested approach for AU Core looks good. I have two more suggestions:

1. At step 3. when checking for the presence of a DAR code (coding.code is 'unknown'), can we add a condition that checks coding.text is not null OR the element's minimum cardinality = 0? If TRUE, then the test should FAIL.
2. In the Missing Data and Suppressed Data Tests, can we add to the introduction something about "These tests check the capability to support the DataAbsentReason extension, and the “unknown” concept code from the DataAbsentReason Code System". It is intended to support manual verification (or other testing mechanism) to fully test conformance to AU Core Missing Data and Suppressed Data requirements."?

Please let me know if you have feedback on the above. Cheers :) cc: @heathfrankel

[StephThaoOng]

@projkov thanks, the missing data tests work well however when run against the Sparked server the results reported seem to be duplicated as shown below:

Secondly, there is a request to include a check for the presence of data suppressed using the code "masked" i.e.

"extension": [
      {
        "url": "http://hl7.org/fhir/StructureDefinition/data-absent-reason",
        "valueCode": "masked"
      }
    ]

and

"coding": [
      {
        "system" : "http://terminology.hl7.org/CodeSystem/data-absent-reason",
        "code" : "masked",
        "display" : "Masked"
      }
    ],

Can you please look into how these can be implemented? Thanks :)

[projkov]

@StephThaoOng Hello Steph

Thank you for the review!

1. It looks like we need to add a filter to remove duplicated resources for validation. I'll take care of it.
2. Yes, it makes sense to add a check for masked items.

I'll do it soon. Thanks!

[StephThaoOng]

thanks @projkov By the way, for context this is how the team are reporting on testing coverage of missing and suppressed data to the TDG and FHIRWG:

So we will use:

• presence of DAR extension with valueCode = "unknown" to report on Missing Data - non-coded elements testing
• presence "unknown" code from the DAR CodeSystem to report on Missing Data - coded elements testing
• presence of DAR extension with valueCode = "masked" to report on Suppressed Data - non-coded elements testing
• presence "masked" code from the DAR CodeSystem to report on Suppressed Data - coded elements testing