Business case and logic flow for multiple options of sending CSV messages to the UP

[dylansmith-Nava]

User Story

As a ReportStream teammate, I want to understand the business case, business value, and cost/difficulty of implementing various options of CSV schemas in the UP, so we can better consider how we might support current CSV senders to send messages in the UP, potentially with multiple conditions.

Background

Owing to the CSV Support in the UP MVP Product Brief, some 50% of current COVID pipeline Senders send messages in CSV format. As we transition these Senders to the UP, we are considering various strategies to both 'meet users where they are' while helping to support ReportStream values and key OKRs of allowing organizations to send multiple conditions. There are a two main options related to supporting a CSV schema in the UP, and which condition(s) we should allow.

Option 1: Only support the COVID-19 condition for CSV submission in the UP

• The schema we have right now works for the CSV Senders in the COVID pipeline now, and we’re just migrating those Senders
• Pros: Likely faster implementation
• Cons: Not inherently designed for longevity

Option 2: Supporting COVID, MPox, Lyme, and HIV conditions for CSV submission in the UP

• Pros: All these conditions have LIVD table lookups, so hopefully won't be a large change to the CSV format
• Cons: Needs research to understand if more column headers are needed

Option 3: Supporting all single analyte and multiplex CSV submission in the UP

• We are planning for longevity, and need to accommodate multiple conditions from the get-go
• Pros: We’d be building a CSV format for the UP that has the most longevity going forward
• Cons: Likely longer build time

Risks/Impacts/Considerations

• The national COVID-19 flat file is somehow not sufficient enough to base our UP CSV schema off of, and the work requires another file to use
• Supporting a minimum suite of conditions is not viable even in the med-long term, because of the further research it may take to support more conditions

Open Questions

• What needs to be added to the national COVID-19 flat file to support Option 2 and 3? How many column headers and which one(s)?
• What would successful Option 2 and Option 3 CSV format look like?
• How will we accommodate multiplex results? How will we make sure that these results are reported together?
• Can we just use the SimpleReport CSV format? Would anything need to change to accept this format?

Links

• Product Brief - CSV Support in the UP MVP
• [CSV>FHIR Research] (https://docs.google.com/document/d/1mk06z-gZa9YAJqYtuZ6aQ3hW3iWFFzsVKNfgYboH7wQ/edit)
• CSV in UP MVP Requirements
• COVID 19 CSV Schema
• CSV->FHIR Formatting & Gap Analysis #12043
• CSV File Formats

Acceptance Criteria

• [x] Proposed CSV schema for Senders to use to submit to the UP for Option 1, 2, and 3
• [x] Business case materials that document the logic flow from Sender sending a CSV to ingestion, validation, enrichment, and output of these messages to a Receiver
• [x] Graphic flow representation of the business logic flow (in Mural)

Out of Scope

• Time and scoping estimate for how long implementation will take for the agreed-upon CSV schema

[dylansmith-Nava]

Closing this ticket and breaking it down into further smaller tickets to tackle individually. Ticket is too big and needs to be closed. However, made large progress on the product brief and option 3 CSV schema, and made some progress on multiplex use case, validation strategy, supporting statistics, and communication strategy (which is a blocker). Will close this ticket and make separate tickets to pursue each of these. CC @jillian-hammer