openmeteo / enhydris

A database with a web interface for the storage and management of hydro/meteorological measurements and time series
GNU Affero General Public License v3.0
18 stars 11 forks source link

Meteoview2 stores Europe/Athens time instead of converting to UTC #503

Closed aptiko closed 1 year ago

Shrhawk commented 1 year ago

When I select UTC in case of Meteoview2, it stores the data_timezone in Etc/GMT which is equal to UTC as per the following link. Since, there are no differences then why do we need to convert it?

aptiko commented 1 year ago

@Shrhawk I'm not certain I understand the question correctly. The Meteoview2 API provides the data in Europe/Athens (actually this is probably configurable for each station, but all the stations we work with are in Greece). But in Enhydris we store data in UTC. So we have to convert from the time zone of the data to UTC. data_timezone is the time zone of the data in the API, not the time zone of the data in the database, which is always UTC.

There is also the field Gentity.display_timezone, but this does not have to do with how the data is stored in Enhydris, only with how it is displayed. The Telemetry app does not display any data, it merely downloads it from an API and then stores it in the database, therefore it's not concerned with display_timezone.

Now if the Meteoview2 API has an option with which to select in which time zone we want to get the data (like addUPI does), then certainly we must get the data in UTC and store it as is, without conversion, and we don't need to show the data_timezone field to the user. (But I'm unaware of any such option.)

Shrhawk commented 1 year ago

@aptiko I understand the flow that Meteoview2 API provides the data in Europe/Athens and we have to convert it to UTC and store it to the database. Can you tell me where exactly we are storing the data to the database?

Also, I was checking get_measurements(), does this method has anything to do with the timezone? If yes, then where are we using it?

aptiko commented 1 year ago

@Shrhawk Here is where get_measurements() is used.

The second statement, in line 103, isn't important for our case; it will usually return exactly what it received. But if there are more than one records in the same minute, it will remove all but the first. (This is explained in the get_measurements() documentation.)

The final statement, append_data(), is the one that will store the data in the database, as explained in its documentation. It looks to me that this is the the one that will convert the timezone if necessary.

The code seems correct. It's possible I've misunderstood something and that the system works as intended. Have you been able to reproduce this issue?