search

HicServices / RDMP

Research Data Management Platform (RDMP) is an open source application for the loading,linking,anonymisation and extraction of datasets stored in relational databases.
https://github.com/HicServices/RDMP#research-data-management-platform
GNU General Public License v3.0
36 stars 16 forks source link

RemoteAttacher unit tests timezone sensitive, underlying bug? #1839

Open jas88 opened 4 months ago

jas88 commented 4 months ago

We have an issue in the RemoteAttacher date logic regarding timezones - it's calculating with local time rather than UTC, and what that timezone is depends on the server in question. Ideally need to switch the date handling in RemoteAttacher to use UTC for everything (DateTime.UtcNow not DateTime.Now in C#, GETUTCDATE() not GETDATE() on MS SQL, UTC_DATE() not DATE() on Oracle...)

This means CI occasionally fails depending on the time of day - the Github VMs are on Pacific time IIRC, so about one-third of the time the local date is a day behind the UTC date, causing an off-by-one which breaks those unit tests.

Not the first bit of RDMP to hit this; there's some older code that spent a while with a "Do not run this test late at night" label on it since it would fail for similar reasons between 11pm and midnight UK time six months of the year...

Originally posted by @jas88 in https://github.com/HicServices/RDMP/issues/1838#issuecomment-2126085125

JFriel commented 4 months ago

@jas88 do you have an example of these failing tests? From a look at the code, I'd guess it's to do with trying to load 'future' data?

jas88 commented 4 months ago

Past, but not always far enough in the past - "2 days ago" sometimes becomes "1 day ago" and causes the time window tests to fail ("expected 2 rows but got 3"). It showed up on the Dependabot PR for Microsoft.Test.Sdk, then they passed when manually re-run a few hours later, because "now" was the same date for both timezones.

Maybe I'll try a specific test for that with a time/date just before and after midnight to catch it reliably.

jas88 commented 4 months ago

@JFriel example here - failed around 6pm Pacific Time (where I think Github's VMs live), early morning the next day UTC. https://github.com/HicServices/RDMP/actions/runs/9294965423/job/25581113234#step:12:1555