noi-techpark / bdp-commons

GNU Affero General Public License v3.0
2 stars 12 forks source link

As a Open Data Hub maintainer I want understand why the meteorological data is missing since 01.12.2022 #603

Closed dulvui closed 1 year ago

dulvui commented 1 year ago

Meteorological data is missing for the types 'air-temperature-min', 'air-temperature-max' for the stations with origin 'EURAC', 'meteotrentino', 'SIAG'. For stations with origin EURAC also data with the type 'precipitation' is missing since that date.

Because the last timestmp for all this data is "2022-12-01 00:00:00.000" I assume that it is a central problem with either the Open Data Hub writer or the database.

Here some queries to see the data with the issue:

select * from measurement m
join "type" t on t.id = m.type_id
join station s on m.station_id = s.id
where t.cname in ('air-temperature-min','air-temperature-max')
and s.origin in ('EURAC','meteotrentino','SIAG')
order by "timestamp" desc
select * from measurement m
join "type" t on t.id = m.type_id
join station s on m.station_id = s.id
where t.cname in ('precipitation')
and s.origin in ('EURAC')
order by "timestamp" desc
dulvui commented 1 year ago

@rcavaliere The datatypes "air-temperature-min", "air-temperature-max" come from the EURAC data collector, also for the stations of BZ and TN. And it seems that in the EDP the data for the year 2023 is still missing and that's why we see this problem. Do you know how often the data in the EDP gets updated?

rcavaliere commented 1 year ago

@dulvui you should check in the correspondent Data Collector, but if I remember well the two provinces provide us real-time data every 5 / 10 minutes. EURAC has longer update time

dulvui commented 1 year ago

@rcavaliere It seems to me that theres no data on the data provider side for the year 2023. On their website the period goes until 2022-12-31 image https://edp-portal.eurac.edu/discovery/15e47115-de56-44b3-a316-03b699515436

So we need to wait until they put new data for the year 2023 into the EDP

rcavaliere commented 1 year ago

@dulvui this is correct, I am more concerned about the real-time data provided by the Provinces, which should be real-time e non statistical (annual) elaborations, as in the case of EURAC!

dulvui commented 1 year ago

@rcavaliere The realtime data from the other providers is working fine. This problem affects only the statistical data coming from EURAC and since the EURAC data collector reuses the stations of the data collectors of the provinces it seems that the data is missing on their side.