Closed DanielSass closed 1 month ago
As part of this we should ensure the following: We should make sure that any results being sent to the CSV pipeline should also map to values here: https://github.com/CDCgov/prime-reportstream/blob/1ffae4ca0b04cd0aa9f169e26813ecd86df71bb5/prime-router/src/main/kotlin/metadata/Mappers.kt#L768 otherwise you get the following error from ReportStream that is not obvious to users uploading their results: RS CSV API Error 400 Response:
{
"id" : null,
"submissionId" : 4261955,
"overallStatus" : "Error",
"timestamp" : "2024-07-11T20:43:28.954Z",
"plannedCompletionAt" : null,
"actualCompletionAt" : null,
"sender" : "",
"reportItemCount" : null,
"errorCount" : 1,
"warningCount" : 0,
"httpStatus" : 400,
"destinations" : [ ],
"actionName" : "receive",
"externalName" : "",
"reportId" : null,
"topic" : null,
"bodyFormat" : "",
"errors" : [ {
"scope" : "item",
"indices" : [ 1 ],
"trackingIds" : [ "42daf9c8-9deb-4636-a040-380bde2b44cd+1+20240711" ],
"field" : "test_result (test_result)",
"message" : "The code '' for field test_result (test_result) is invalid. Reformat to HL7 specification.",
"errorCode" : "INVALID_MSG_PARSE_CODE"
} ],
"warnings" : [ ],
"destinationCount" : 0,
"fileName" : ""
}
Since deploying #7897, I'm checking what folks have submitted as codes that we don't support using the following query:
traces
| where message has "Unsupported SNOMED result code:"
| parse message with * "Unsupported SNOMED result code: " SNOMEDCodeEntered
| extend SNOMED = split(SNOMEDCodeEntered, ":")
| mv-expand SNOMED to typeof(string)
| summarize occurrences=count() by SNOMED
And I'm seeing, these results
File validation for the bulk result uploader was a contributing factor to the bug described in #7897.
Requested update:
Adjust the validation for the
test_result
field in the bulk uploader to be more stringent./backend/src/main/java/gov/cdc/usds/simplereport/validators/CsvValidatorUtils.java#L273-L275
Rather than allowing anything that looks like a snomed (i.e. the regex currently in place), we should validate against a list of specific snomeds that we support and can effectively map in the fhir converter (which should include any snomeds in the covid or hiv livd tables).
The SNOMEDs we allow should be the same ones being explicitly mapped to normal/abnormal flags in #7897