search

openaq / openaq-fetch

A tool to collect data for OpenAQ platform.
MIT License
84 stars 39 forks source link

EEA UTC+0 local time stamp problem #386

Open maxgrossman opened 7 years ago

maxgrossman commented 7 years ago

When conducting dry runs in reference to #383, I noticed a problem with EEA sources that have a local UTC offest of 0.

In this case, some of Portugal's records located in zome AZOST (Azore Summertime which has an offset UTC+0) generated local timestamps like the below:

2017-09-20T06:00:00Z
2017-09-20T07:00:00Z
2017-09-19T23:00:00Z
2017-09-20T00:00:00Z
2017-09-18T16:00:00Z

I did some very minimal reworking of how the eea adapter makes dates and if we just make the local time value an inline conditional the issue appears to be fixed. That is, changing local from:

local: localDate.format()

to:

local: new RegExp(/00:00/).test(localDate.format('Z')) ? localDate.format('YYYY-MM-DDTHH:mm:ssZ') : localDate.format()

cc @olafveerman @jflasher

majesticio commented 1 year ago

@russbiggs this is a persistent bug from what I can tell that is affecting many EEA entries in our database. Localtime format looks the same as UTC. Data validation should prevent this from happening, I get errors if I run an adapter where time is formatted incorrectly. For example look at the times on https://api.openaq.org/v1/measurements?country=PT

majesticio commented 1 year ago

Perhaps this is only true where the offset = +0? In which case maybe its not a bug? Need a second pair of eyes on this