HHS / Head-Start-TTADP

OHS TTA Hub data system. ACF SO, Dana Jones-Quartey. Tech lead: Krys Wisnaskas
Main: https://ttahub.ohs.acf.hhs.gov/
21 stars 7 forks source link

[TTAHUB-3451]format comlog comdate history #2467

Closed hardwarehuman closed 2 weeks ago

hardwarehuman commented 2 weeks ago

Description of change

Improperly formatted communicationDate values were making their way into CommunicationLogs.data. This fixes the history and produces all valid dates. There's one date in the year 3034, but that's the date that was input by the user.

How to test

You can see what all the updates will be just by inspecting SELECT * FROM comdate_corrections;

Issue(s)

Checklists

Every PR

Before merge to main

Production Deploy

After merge/deploy

GarrettEHill commented 2 weeks ago

Please also update src/queries/dataRequests/user/communication-logs.sql as its logic around the date is now not needed.

GarrettEHill commented 2 weeks ago

Looks good, I almost wonder if it could be simplified any, but I see you had to deal with weird cases. Like where you don't have the starting portion of the year ie 24 = 2024.

Some AI code: -- Get all the bad cases. SELECT * FROM "CommunicationLogs" WHERE data->>'communicationDate' !~ '^(0[1-9]|1[0-2])/(0[1-9]|[12][0-9]|3[01])/([0-9]{4})$';

-- Sample update (obv we cant use as is) UPDATE "CommunicationLogs" SET data = jsonb_set(data, '{communicationDate}', '"01/01/1970"') WHERE data->>'communicationDate' !~ '^(0[1-9]|1[0-2])/(0[1-9]|[12][0-9]|3[01])/([0-9]{4})$';

Something like this might be used to handle the weird date cases (again AI generated) but might be what you have or close to it:

UPDATE "CommunicationLogs" SET data = jsonb_set( data, '{communicationDate}', to_jsonb( CASE WHEN data->>'communicationDate' ~ '^(0[1-9]|1[0-2])/(0[1-9]|[12][0-9]|3[01])/([0-9]{2})$' THEN regexp_replace(data->>'communicationDate', '/([0-9]{2})$', '/20\\1') ELSE data->>'communicationDate' END ) ) WHERE data->>'communicationDate' !~ '^(0[1-9]|1[0-2])/(0[1-9]|[12][0-9]|3[01])/([0-9]{4})$';

What you have is good, don't use the above as it makes this harder to follow that what you already have. This is a one and done migration not something that needs to be for preferment.

GarrettEHill commented 2 weeks ago

Please also update src/queries/dataRequests/user/communication-logs.sql as its logic around the date is now not needed.

This still needs to be addressed