qe-collaborative-services / 1115-hub

New York 1115 Medicaid Waiver Hub
https://qe-collaborative-services.github.io/1115-hub/lib/ahc-hrsn-elt/screening/
GNU Affero General Public License v3.0
1 stars 10 forks source link

Invalid dates from CSV tests for Healtheconnections #208

Closed ghoward9 closed 2 months ago

ghoward9 commented 2 months ago

I am seeing FHIR with invalid dates coming from TechBD to the NYeC FHIR Validator endpoint from the QE Healtheconnections today for all CSV tests. Other QEs that were sent the same CSV tests are PASSing with valid FHIR.

An example is attached for CSV testcase1 form Healtheconnections. hec7_17.json

alan-francis commented 2 months ago

Hi @ghoward9 , according to the business rules listed in the omnibus rules that we follow here, we are validating the RECORDED_TIME against the format YYYY-MM-DDTHH:MM:SS.000Z. Currently we are passing those values with a warning as long as the field is not empty as shown below. This might be a strict validation in the FHIR validator that we are sending this FHIR JSONs.

1115 Recorded_Time_Rule

The generated FHIR from Healtheconnections had the incorrect date-time format as we can see from the TechBD self service portal.

Flat-File-CSV-Data-Quality

@reeager @shah @raphaelvp @ratheesh-kr

alan-francis commented 2 months ago

@reeager @ghoward9 : Should we need any changes in the business rules? We can update the code to make the message more clear from the TechBD portal side.

fredharmonbcbsscgit commented 2 months ago

Hi Alan,

I took a look at this week's test for HEC as well as their last 8 week's worth of CSV tests to see if we had started changing the way date was sent in the CSV. It appears like we have been sending the dates in the same format for everyone for many months now. In testcase 1 from 7/17, the date in RECORDED_TIME looks like "2023-02-15T10:58:12.722227+02:00"

What seems odd is that we only seem to have this problem for HEC. For example, yesterday we sent to HealtheLINK for test case 1 the RECORDED_TIME of "2024-01-19T11:36:22.471340+05:00" and that test case converted to FHIR on your side just fine and passed our side. Another example from yesterday for Rochester was test case one with a RECORDED_TIME of "2023-04-08T10:29:33.945212+05:00" that came back to us fine.

Is there any chance HEC is running on a different code branch on the TechDB side? Is there some special processing for HEC perhaps?

Thanks

Fred

alan-francis commented 2 months ago

Hi Fred,

I have attached the image of one of the screening CSVs that was sent from HEC. I think some sort of formatting in the date is happening as a result of which the date get updated into an unwanted format. From the image we can see that the recorded time is - 2023-02-15 10:58:12.7222270.

image_2024_07_18T13_30_49_887Z

@shah @raphaelvp @ratheesh-kr

drubacha commented 2 months ago

Hi All, HeC was previously taking the date for recorded time and storing in a SQL database column as Datetime2. This caused the formatting change when we recreated the csv file to send to you. The column has been updated to be a varchar so whatever you send to us is now what we will send back to you.

raphaelvp commented 2 months ago

As per the above comment, closing the issue.