search

noi-techpark / bdp-commons

GNU Affero General Public License v3.0
2 stars 12 forks source link

As air quality expert, I would like to check if certain air quality data from the Province of Bolzano is correctly imported in the Open Data Hub #668

Open rcavaliere opened 1 month ago

rcavaliere commented 1 month ago

The BrennerLEC partners have noticed that specifically on the ML5 air quality stations of the Province ("A22 Egna - A22, corsia sud km 103") we have frequent data holes, which should not appear.

Check for example: link

Generally, the open data (period = 3600) the data seem to have timestamp UTC + 2, while it should UTC +1.

Affected Data Collectors to check: https://github.com/noi-techpark/bdp-commons/tree/main/data-collectors/environment-appa-bz-tenminutes https://github.com/noi-techpark/bdp-commons/tree/main/data-collectors/environment-appa-bz-opendata

rcavaliere commented 3 weeks ago

@dulvui today I got the information that the Data Provider (APPABZ) has fixed an issue, now the data should be available always as UTC+1. Can you please check?

dulvui commented 3 weeks ago

@rcavaliere For opendata I simplified now the timestamp conversion and it should be correct now. The sync triggers every morning at 10, so lets see tomorrow if the issues are solved.

dulvui commented 3 weeks ago

There is still missing data for some days. I will setup now some logging to understand better if we loose the data or if the API doesn't update the data

dulvui commented 1 week ago

@rcavaliere I just saw now that there might be a problem with the data provider, since the values for Egna are -1, and in that case values are not valid and get ignored by the data collector. Here the url for the station at Egna http://dati.retecivica.bz.it/services/airquality/timeseries?station_code=ML5

Here an example response with -1 value, since this will change if called on another day.

[{"DATE":"2024-06-23T12:00:00+01:00","SCODE":"ML5","MCODE":"CO","TYPE":"1","VALUE":-1},{"DATE":"2024-06-23T12:00:00+01:00","SCODE":"ML5","MCODE":"NO2","TYPE":"1","VALUE":-1}]

I added now specific logging for the case that the value is -1, so I can see for which stations/data types this happens