CA Round-Trip Testing

[JohnNKing]

Story

As a PHL or HCO, so that we're confident the various branches of our workflows will work under ETOR, we need to test various scenarios of NBS results between partner systems (e.g. EHR, LIMS).

Pre-conditions

• Test planning has been completed
• Data gap analysis efforts have been completed and the test scenarios have been defined
• PHL and HCO can send and receive results messages to each other via ReportStream
• Connectivity and integration testing have been completed for both HCO and PHL

Acceptance Criteria

• [ ] PHL has released test results based on the scenarios
• [ ] The HCO successfully processed those test results

Tasks

• [x] Confirm with partners that no PII/PHI will be used during testing and that test messages can be later published in our public repo
• [ ] PHL executes the results tests in the test/staging environment
• [ ] Ensure hospital and lab have agreed upon a means to validate data integrity (e.g. shared screenshots of results report as displayed to clinician in EHR)
• [x] We've saved the HL7 messages used for testing to later create automated end-to-end testing
• [ ] Draft subsequent stories for automated testing to avoid regressions in the future (e.g. how to we ensure UCSD-specific transformations continue to work as code changes occur)
• [ ] We've revised internal test messages based on test results that were actually submitted by the partner (i.e. our collection of samples messages have been updated)
• [x] Track down cause of 3 undelivered sample messages on 10/16
  • [x] 2 were related to ORC-21-10
  • [x] We need to track down the cause of the third - @basiliskus , @GilmoreA6
• [x] Determine if ORC-21-10 data issue was an artifact of testing, or if modification of the filters will be required
• [ ] Update vocab in the results profile based on the recently identified issue with unsats

Test Case Scenarios (link to master test scenarios sheet)

• [ ] 1. In Range (Normal) Results
• [ ] 2. a. Out of Range Results (GALT, BIO, SCID, Hypothyroidism, CAH, GAA, MPS-I, SMA, MPS-II, GAMT)
• [ ] 2. b. Out of Range Results (CF, Hemoglobinopathies)
• [ ] 2. c. Out of Range Results (Amino Acid disorders, Acylcarnitine disorder)
• [ ] 3. a. Indeterminate Results (SMA, SCID, GAA, MPS-I, MPS-II)
• [ ] 3. b. Indeterminate Results (Blood Transfusion)
• [ ] 3. c. Indeterminate Results (CF Incomplete)
• [ ] 4. Mixed Results - In Range, Indeterminate, Out of Range, Unsatisfactory
• [ ] 5. Additional Sequencing Performed for CF, MPS I, MPS II, Pompe, and ALD
• [ ] 6. Second Tier/Additional Initial Screen Testing Performed for CAH
• [ ] 7. a. Out of Range Results for SCID - Initial Screen
• [ ] 7. b. SCID Confirmatory Results - Unsolicited Message in Follow Up
• [ ] 8. Unsatisfactory Specimen

Definition of Done

• [ ] Documentation tasks completed
  • [ ] Documentation and diagrams created or updated
  • [ ] ADRs (/adr folder)
  • [ ] Main README.md
  • [ ] Other READMEs in the repo
  • [ ] If applicable, update the ReportStream Setup section in README.md
  • [ ] Threat model updated
  • [ ] API documentation updated
• [ ] Code quality tasks completed
  • [ ] Code refactored for clarity and no design/technical debt
  • [ ] Adhere to separation of concerns; code is not tightly coupled, especially to 3rd party dependencies
  • [ ] Code is reviewed or developed by pair; 1 approval is needed but consider requiring an outside-the-pair reviewer
  • [ ] Code quality checks passed
• [ ] Security & Privacy tasks completed
  • [ ] Security & privacy gates passed
• [ ] Testing tasks completed
  • [ ] Load tests passed
  • [ ] Unit test coverage of our code >= 90%
• [ ] Build & Deploy tasks completed
  • [ ] Build process updated
  • [ ] API(s) are versioned
  • [ ] Feature toggles created and/or deleted. Document the feature toggle
  • [ ] Source code is merged to the main branch

Research Questions

• Optional: Any initial questions for research

Decisions

• Optional: Any decisions we've made while working on this story

Notes

• Optional: Any reference material or thoughts we may need for later reference

[GilmoreA6]

For Task "Track down cause of 3 undelivered sample messages on 10/16" - -We've confirmed that two of the 3 messages that didn't route due to absence of ORC-21-10 that is used for filtering reports on the RS side and for identifying messages for applying the USCD transform in ETOR-TI. -The third message did not route due to an exception when attempting to parse the fhir bundle in RS for element Bundle.entry.resource.ofType(Observation).ValueQuantity.value of "065.88" where exception " ca.uhn.fhir.parser.DataFormatException: HAPI-1861: Failed to parse JSON encoded FHIR content: Invalid numeric value: Leading zeroes not allowed at [line: 1, column: 137695]" was thrown. The problematic value was converted from the OBX-5 value from the OBX segment: OBX|1|NM|38473-5^17-Hydroxyprogesterone^LN|1|0065.88|nmol/L|<85|N|||F|||202410150557

[scleary1cs]

"Determine if ORC-21-10 data issue was an artifact of testing, or if modification of the filters will be required"

We received a verbal and email confirmation that this data issue will not effect Prod.

"Two of the test messages are missing ORC-21.10. This value is critical for routing - can CDPH confirm it will always be populated in production? Sumitha, can you please answer this question? – Yes it will be always populated in PROD." Subject = "Vocabulary Clarifications for ETOR Intermediary"

[kwright-bio]

Confirmed in SME call that 2b was ingested into the system. Can mark as complete once receiving a PDF from UCSD. TCs 1, 7A, and 4 have had successful ingestion into UCSD but prior to the mu fix being implemented. These messages can likely be redropped and resent to UCSD and get confirmation the mu character looks good in final PDF exports for these (confirm this plan with team / HCO)

[kwright-bio]

Also a note that the CDPH generated results being used in testing are from a custom-built test result generator and not their SpecimenGate system.

[scleary1cs]

@kwright-bio and @dbgolson happy to circle back here and update the test cases based on what was just shared at Sprint Review.

[kwright-bio]

Added the newly created Indeterminate test cases. Note that 2b's comparison uncovered an issue where likely they have not mapped a code correctly.