Structural Validation of NBS Orders and Results

[JohnNKing]

Story

Sentence

As a hospital (or lab), to ensure my NBS order (or result) can be accepted and processed, I need ReportStream to perform structural validation on the incoming order (or result).

Pre-conditions

• This effort is constrained to the HL7 version (2.5.1) and message types (e.g. ORM, ORU) that ReportStream currently support.

Acceptance Criteria

• [x] If a structurally invalid HL7 order/result is received by the ReportStream service, an error is returned to the hospital communicating that it was invalid.
• [x] We have a documented list of HTTP response codes that ReportStream may return for different structural errors that can be shared with our partners.

Tasks

• [ ] Review any documentation that RS has for HTTP responses
• [ ] Test messages we currently have to ensure we get the responses we expect

Research

• [ ] Research work needed to complete the story
• [ ] What are the differences and similarities between the states (AL, LA, and CA)?
• [ ] Is validation performed implicitly when RS does its transformations? If it does, who ultimately needs to create the error message for consistency? (We need a consistent mechanism for generating errors, regardless of whether they occur in RS or TI.)
• [ ] What errors would prevent the order from being ingested by the LIMS?
• [ ] What standards should validation be done against (FHIR R4 & USCDI v1 (current ONC rule) or v3 (US Core v6, new ONC rule))?
• [ ] What specification versions do Epic and Cerner's software support?
• [ ] Can we reuse the NIST validator? Does that validate both structure and business values? Compare to HAPI FHIR Validator.
• [ ] Does scoping validation as laid out in the draft roadmap make sense? (e.g. structure, field presence, data type, specific field type) Or is there a better way to broach this?
• [ ] Is there any way of supporting synchronous validation via a plugin/dependency to ReportStream that we provide?
• [ ] How would we prevent logging of PII/PHI?

Engineering

• [x] Document what RS is validating

Findings

Here's what we found RS validates for ORM and ORU. More details in this spreadsheet

• ORM message
  • Required segments: MSH, PID, ORC, OBR, OBX
  • Required fields:
  • All required segments require field 1
  • MSH: 1, 2, 6, 9, 10, 12 (6 doesn’t throw any error or warning, it just doesn’t deliver)
• ORU message
  • Required segments: MSH, PID, OBR
  • Required fields: same as ORM

There are 3 different stages in the flow where validation and errors happen:

1. Sync/HTTP Response: this is where errors related to HL7 parsing (HapiFHIR?) happen. Fields required to parse the message are validated here
2. Sender => RS/Hl7 => FHIR: this is where errors in the conversion from HL7 to FHIR happens. Field type validation happen here
3. RS => Receiver/FHIR => HL7: errors in conversion from FHIR to HL7. Required segments in the HL7 are validated here

Definition of Done

Engineering

• [x] Threat model updated
• [x] Code refactored for clarity and no design/technical debt
• [x] Adhere to separation of concerns; code is not tightly coupled, especially to 3rd party dependencies.
• [x] Source code is merged to the main branch.
• [x] Unit test coverage of our code >= 90%
• [x] Code is reviewed or developed by pair; 1 approval is needed but consider requiring an outside-the-pair reviewer.
• [x] Build process updated
• [x] Feature toggles created and/or deleted. Document the feature toggle.
• [x] Source code documentation created when the code is not self-documenting.
• [x] API documentation generated
• [x] Code quality checks passed
• [x] Security & privacy gates passed
• [x] Load tests passed
• [x] Documentation and diagrams created or updated
• [x] API(s) are versioned
• [x] Debug logging

Research Questions

Decisions

• Optional: Any decisions we've made while working on this story

Notes

[JohnNKing]

Might this leverage ReportStream quality filters? (Discussed today during standup)

[JohnNKing]

Use this story to ensure we verify existing RS ORM => FHIR conversion?

[scleary1cs]

This Story helps complete the requirement: MVP-REQ-04 - Each intermediary will create an MVP with the capability to support when the facility receives the entire order, and the intermediary is responsible for validating and transforming the order for PHL

[scleary1cs]

This Story helps complete the requirement: MVP-REQ-05 - Each intermediary will create an MVP wherein the system accepts all order-related information from the facility. The intermediary is responsible for receiving the order, performing validation, and carrying out transformations for PHL LIMS

[scleary1cs]

This Story helps complete the requirement: MVP-REQ-15 - Intermediaries will establish, utilize, and validate a common set of data elements

[scleary1cs]

This Story helps complete the requirement: MVP-REQ-36 - Each intermediary will have the same validations performed for all data exchange formats (e.g, NIST)

[Adrian-Brewster]

What validation is ReportStream doing currently? How can we leverage the message validator tool on Reportstream's website?

[basiliskus]

The AC were completed in other stories so there's no new work to do here. The Research tasks are related to integration with partners and not related to this story

[scleary1cs]

@Adrian-Brewster Can we convert your comment into it's own story?

[scleary1cs]

Above research tasks (new engineering stories need to be made):

-Can we reuse the NIST validator? Does that validate both structure and business values? Compare to HAPI FHIR Validator.

-Is there any way of supporting synchronous validation via a plugin/dependency to ReportStream that we provide?

-How would we prevent logging of PII/PHI?

[scleary1cs]

Above research tasks (new research stories need to be made):

• What are the differences and similarities between the states (AL, LA, and CA)?

• Is validation performed implicitly when RS does its transformations? If it does, who ultimately needs to create the error message for consistency? (We need a consistent mechanism for generating errors, regardless of whether they occur in RS or TI.)

• What errors would prevent the order from being ingested by the LIMS?

• What standards should validation be done against (FHIR R4 & USCDI v1 (current ONC rule) or v3 (US Core v6, new ONC rule))?

• What specification versions do Epic and Cerner's software support?

• Does scoping validation as laid out in the draft roadmap make sense? (e.g. structure, field presence, data type, specific field type) Or is there a better way to broach this?

Will be discussed at research collab session on 03/01

[JohnNKing]

Programmer's Guide has been updated: https://docs.google.com/document/d/19xii5OlERxziIRfHftVzz8o1u9AHIbDdJlOb3B3mVVw/edit