HL7 support

[JoeBatt1989]

Summary of problem

AI Applications want the ability to be able to send HL7 messages back to clinical systems (i.e RIS, CRIS) to support worklist prioritisation.

Summary of proposed change

To facilitate the integration and transmission of HL7 messages back to MONAI for onward delivery to clinical systems, modifications are needed in both MIG and MWM which covers:

• The creation of a HL7 server which receives and transmits HL7 messages
• The re-identification of PII data in the HL7 message
• The introduction of a new event to notify Workflow Manager when results are returned
• Logic which allows Workflow Manager to dispatch tasks even when artifacts may not have been returned
• Management of RIS connections

Addition of a modality type to artifacts object

In order to be able to manage multiple results of different or same modality returned, the MWM and MIG must be able to support a new modality type field. This will allow the platform to support multi-dicom results of different types (SEG, RT, SC ect) and HL7 (and can be extended further to support FHIR).

            "artifacts": {
                "input": [
                    {
                        "name": "input-dicom",
                        "value": "{{ context.input.dicom }}",
                        "mandatory": true,
                    }
                ],
                "output": [
                    {
                        "name": "encapsulated-pdf",
                "type": "DOC" 
                    },
                    {
                        "name": "segmentation",
                "type": "SEG" 
                    },
                    {
                        "name": "structured-report",
                "type": "SR" 
                    },
                    {
                        "name": "hl7-message",
                "type": "HL7" 
                    }
                ]
            },

By knowing the types of files that we are expecting back, the workflow manager can make a decision whether to continue the task or wait for further results to be returned before dispatching the next task. Currently if any DICOM results are returned then the workflow manager will infer that the task has passed and carry on when in fact the AI application may be sending more results.

Artifacts Returned Event

A new event will be created and will replace the WorkflowRequestEvent logic that has been implemented for remote app execution. The WorkflowRequestEvent was updated to support Workflow continuation by the addition of a WorkflowId and a TaskId. However this will not work when multiple results are returned. The proposal is to replace this with a new event named ArtifactsReturnedEvent.

{
    "correlationId": "",
    "WorkflowInstanceId":"",
    "TaskId":"",
    "Artifacts": [
    {
        "Name": "",
        "Type":"",
        "Location":""
    }]
}

Logic to manage artifacts and task dispatches independently

Upon receiving an artifactsReturnedEvent the WorkflowManager must add the artifacts to the WorkflowInstance.Task.ArtefactDictionary and dispatch the next tasks accordingly by checking the input artifacts of the tasks from the TaskDestinations array.

The reason for managing these artifacts separately is that results are returned async and there may be a time where meaningful DICOM results are returned but the HL7 never arrives, but in this case you would still want to dispatch the task to export the DICOMs back to PACS and not wait for a HL7 which may never arrive (or vice versa).

Workflow Example

{
    "tasks": [
        {
            "id": "remote",
            "task_destinations": [
                {
                    "name": "export-dicoms"
                },
                {
                    "name": "export-hl7"
                }
            ],
            "artifacts": {
                "input": [
                    {
                        "name": "input-dicom",
                    }
                ],
                "output": [
                    {
                        "name": "segmentation",
                        "type": "SEG"
                    },
                    {
                        "name": "encapsulated-pdf",
                        "type": "DOC"
                    },
                    {
                        "name": "hl7",
                        "type": "HL7"
                    }
                ]
            },
            "input_parameters": null
        },
        {
            "id": "export-dicoms",
            "description": "export dicoms",
            "type": "export-dicom",
            "export_destinations": [
                {
                    "name": "PACS"
                }
            ],
            "artifacts": {
                "input": [
                    {
                        "name": "segmentation",
                        "value": "{{ context.executions.remote.artifacts.segmentation }}",
                        "mandatory": true
                    },
                    {
                        "name": "encapsulated-pdf",
                        "value": "{{ context.executions.remote.artifacts.encapsulated-pdf }}",
                        "mandatory": true
                    }
                ]
            }
        },
        {
            "id": "export-hl7",
            "description": "export hl7 message",
            "type": "export-hl7",
            "export_destinations": [
                {
                    "name": "RIS"
                }
            ],
            "artifacts": {
                "input": [
                    {
                        "name": "hl7",
                        "value": "{{ context.executions.remote.artifacts.hl7 }}",
                        "mandatory": true
                    }
                ]
            }
        }
    ]
}

segmentation result returned

• Workflow Manager will add the artifact to the task dict in the Workflow Instance
• Workflow Manager will update the Task to “Partial Pass”
• Workflow Manager will not dispatch any tasks as all input artifacts are not returned

encapsulated pdf returned

• Workflow Manager will add the artifact to the task dict in the Workflow Instance
• Workflow Manager will check if all outputs have been returned, if not then status will remain as “Partial Pass”
• Workflow Manager to dispatch export-dicoms task as all input artefacts have been returned

HL7 message returned

• Workflow Manager will add the artifact to the task dict in the Workflow Instance
• Workflow Manager will check if all outputs have been returned, all artifacts are returned and task is marked as “passed”
• Workflow Manager to dispatch export-dicoms task as all input artefacts have been returned

If an artifact is not returned within the timeout period set but has already been marked as “Partial Pass”, then Workflow Manager will mark the Task as “Partial Fail” and the issue will be shown in the issues table.

RIS Integration

Option 1

MIG changes

New DB and endpoints to be created in MIG to support the integration of RIS. Endpoints must support create, update and delete (copy of DestinationApplicationEntity).

{
    "Name": "",
    "HostIp": "",
    "CreatedBy": null,
    "UpdatedBy": null,
    "DateTimeUpdated": null,
    "Port": 80
}

MWM changes

Workflow Manager must check that the name of the export destination is either in DestinationApplicationEntity or in DestinationInformationSystems

Option 2

MIG changes

Add a new type field to the DestinationApplicationEntity table and make the DB name more generic. The type will be an enum and will differentiate DICOM and HL7 destinations. Default will be DICOM

{
    "Name": "",
    "AeTitle": "",
    "HostIp": "",
    "Type": 1,
    "CreatedBy": null,
    "UpdatedBy": null,
    "DateTimeUpdated": null,
    "Port": 80
}

MWM changes

No change

HL7 server to receive results

A HL7 server will be needed in MIG to take results from remote applications. This server will take HL7 messages and do the following:

• Re-identify PII data based on the DICOM metadata of the study sent for inference
• Save the HL7 message to MinIO (.hl7)
• HL7 .net libraries will be leveraged for this (i.e https://github.com/nHapiNET/nHapi)

Export Request Event

Two new task “types” will use the existing queues but will provide information on the type of artifacts that are being exported.

        {
            "id": "export-dicoms",
            "description": "export dicoms",
            "type": "export-dicom",
            "export_destinations": [
                {
                    "name": "PACS"
                }
            ],
            "artifacts": {
                "input": [
                    {
                        "name": "segmentation",
                        "value": "{{ context.executions.remote.artifacts.segmentation }}",
                        "mandatory": true
                    },
                    {
                        "name": "encapsulated-pdf",
                        "value": "{{ context.executions.remote.artifacts.encapsulated-pdf }}",
                        "mandatory": true
                    }
                ]
            }
        },
        {
            "id": "export-hl7",
            "description": "export hl7 message",
            "type": "export-hl7",
            "export_destinations": [
                {
                    "name": "RIS"
                }
            ],
            "artifacts": {
                "input": [
                    {
                        "name": "hl7",
                        "value": "{{ context.executions.remote.artifacts.hl7 }}",
                        "mandatory": true
                    }
                ]
            }
        }

MIG Changes

Based on the “type” attached to the event, MIG will either invoke DICOM SCU or HL7 to send the artifacts attached.

Once done MIG will generate an export.request.complete event. Validation may need to be looked at (TBC)

MWM Changes

Based on the “type” specified in the export task, only relevant artifacts must be specified. The Workflow Manager will know this but checking the output artifact types of a previous task.

Based on the “type” specified in the export task, only relevant export detsinations must be specified. The Workflow Manager must check that these exist and are correct for the export task.

HL7 server to send HL7 artifact

A HL7 server to send the message to an intended destination will be needed. This will be invoked when an export request of type “export-hl7” is received.

Other considerations

On premise execution

Applications that are executed on premise will be responsible for saving their app's HL7 message in a .hl7 format which will be saved to MinIO (same way we save DICOM results). There will be no requirement to pseudonymise and re-identify PII data by the platform as an app executing on premise will have that context.

Documentation will be uplifted to provide guidance to app developers.

Folders

Some applications will generate folders as part of their output. These folders will potentially contain .dcm and .hl7 files. “Folders” will be a valid type for output and input artifacts. In this case for export requests, depending on the “type” of export the Workflow Manager will traverse folders in MinIO to find all .dcm or .hl7 files respectively.

This is currently done for .dcm files and would need extending for .hl7

[ericspod]

CC @onurulgen.

[MMelQin]

The MIG and WMW can blindly cache and send the opaque HL7 messages to any destinations as configured. This issue, however, also puts forward the requirement for the MIG to re-identify the PHI, introducing complexity, IMHO de-id and re-id should not be the responsibility of the MIG. In any case, there are few things to clarify,

• What versions of Hl7, v2.7, v2.8, or v3 or both?
• What message types are expected from the applications? ORU only?
• How exactly the message can convey the "priority" semantics?
• HL7 message contents are regarded as opaque except the PID, PV1, and any patient ID related segments? There are many.
• validation of the message content by the MIG, version specific?
• Point to point, point to multi-point, ack required?

It'll be useful to explain what and how message content can be used to prioritize worklist, as there is no specific message or attribute for it, so

• Is the said "RIS" dictating certain proprietary "code" in the standards define segment and field?
• How is this information known to the application, and with what algorithm and rules can an application, in isolation, set the priority, if a field is identified?
• What are the thoughts on making this generally work for most, if not all, case?

[JoeBatt1989]

@MMelQin thank you for the review. I do understand your concern about bloating the MIG so would be open to discussing other solutions. Currently DICOM pseudo is done via a plugin and HL7 would be done the same way...unless we have a better solution? I will continue to add more details to this ticket as we refine.

Clarification answers below.

• So the version I have been told is 2.4 (although that is just 1 vendor), however they integrate with another HL7 server which manages the version uplift etc depending on the HL7 version supported by RIS. This is something we will have to think about by either creating a seperate service for managing versions using a library like nHAPI. We have health integration engineers at Answer and I plan to grab some time from them to solutionise.
• Yes its for ORU.
• The priorty is defined in the OBR section
• So the exmaples I have been sent have PID (although fairly limited - name, patient ID) and a ZDS section which would need to reflect the StudyUID as well.
• Yes it would need to be validated to check that it is a valid HL7 message and matches an expected version. Versioning is an interesting...this will need a wider discussion.
• Currently export tasks are multi-destination so unless we want to change that then yes we would need to. Youre right we would need to cater for ack, as well as retries (that would be best as we could confirm that the message was received successfully and publish an export.request.complete event). We should be able to leverage an existing open source library to solve this

Useful answers

• So the ORU message is used and from what I can gather is the integration is custom. Meaning the receiving system must action the message accordingly. This would be outside of the scope of MONAI
• Good question, again the integration is custom. I have seen applications have a configurable set of values for prioritisation and the Trusts specify what works for them.
• Yeah agree...I would like us to be as generic as possible to fit as many use cases.

I would think that we could iterate over the delivery of this and break it into logical milestone. For example:

1. Receiving ORU back from an app and saving to MinIO
2. Managing artefacts independently
3. Creation of export-hl7 and export-dicom types
4. Configuration for RIS
5. Send and receive ack and report back to Workflow Manager
6. Re-identification of PII
7. Message version management (or Trusts manage this TBC based on further discussions)

[JoeBatt1989]

Example from docs. https://azmed.notion.site/HL7-ORU_R01-IHE-standardised-message-52a15b0bfc624f34b14899029bbdf77a

[MMelQin]

@JoeBatt1989 thanks for the detailed reply.

A few points to add.

RIS is a generic software component/service, which can be used to drive DICOM modality list, procedure steps, reading worklist, reports, etc, and often has proprietary APIs. So when we say integrating with RIS, it is not directly with the "RIS Service", at least not from my experience, rather it is through the standard HL7 message to/from RIS's own HL7 engine.

ORU is for reporting, and the ORU_R01 is for unsolicited transmission of an observation. Here is also a good page for the message definition. There is no specific defined property for "Priority", and information related to priority has to be embedded in certain property, relying on the RIS/HL7 engine's mapping capability to translate the semantics. The ORU simply provides the observation result, one piece of information in determining the Rad read worklist priority, which itself can be, and always is, institution dependent.

@GreySeaWolf had put forward a similar issue in the MONAI Deploy App SDK board, and has demo'ed sending HL7 message directly from an (MONAI) app, providing coded info as input to worklist priority assessment, at RSNA 2022.

[JoeBatt1989]

Closing this ticket as coverted to being a discussion which is preferred for the Thursday MONAI call. https://github.com/Project-MONAI/monai-deploy/discussions/148