As an Eurac Researcher I want that the weather data provided by the Sensor System Technologies Lab are integrated in the Open Data Hub, in order to be able to retrieve all data I need only by the Open Data Hub.

noi-techpark / bdp-core

Open Data Hub / Timeseries Core

https://opendatahub.com

Other

9 stars 4 forks source link

As an Eurac Researcher I want that the weather data provided by the Sensor System Technologies Lab are integrated in the Open Data Hub, in order to be able to retrieve all data I need only by the Open Data Hub. #254

Closed sseppi closed 1 year ago

sseppi commented 2 years ago

[x] Gruppo FOS develops a data collector that takes the weather data provided by Eurac and stores it in the Open Data Hub (deadline end of June 2022) --> Dedicated Kanban: https://github.com/orgs/noi-techpark/projects/34
[x] NOI checks if the existing data of some weather stations taken from SIAG are not simple duplicates of the new data collector
[x] NOI integrates the new data collector in the ODH infrastructure

Piiit commented 2 years ago

@sseppi @rcavaliere Could you please provide more information about this issue?

sseppi commented 2 years ago

@Piiit till the end of June Gruppo FOS will develop a new data collector that takes the weather data provided by Eurac and stores it in the Open Data Hub. Since we take already the data of some station also from SIAG, the most critical part would be to check that Gruppo FOS doesn't duplicate the stations and stores the data with the source at a measurement level.

rcavaliere commented 1 year ago

PR available, ready to be integrated in the testing environment: https://github.com/noi-techpark/bdp-commons/pull/545 @dulvui can you take care of this please?

dulvui commented 1 year ago

The dc is now running in testing environment with scheduled runs every 12 hours (at 12:00 and 24:00). Tomorrow the data should be visible in the database. Is this all open data? So I will make it publicly available trough Ninja API

rcavaliere commented 1 year ago

@dulvui nice! Yes, should be open data but let's wait that the open data contract with EURAC is signed first. This is followed by @sseppi. In the meantime, let' see if this DC puts down our testing infrastructure :-)

rcavaliere commented 1 year ago

@dulvui I have tested the DC, works perfectly as expected, very good! New measurements are appended to existing stations are requested, new stations are created correctly, existing types reused. Very good work by Gruppo FOS! @sseppi is the contract with EURAC signed? So that we can release the data as open data when we go into production.

sseppi commented 1 year ago

Hi, great news! @rcavaliere no, at the moment the contract isn't signed. Since the reference person is on holiday till the 2nd August. I will contact him again in order to speedup the process.

rcavaliere commented 1 year ago

@dulvui we can put the DC in production!

dulvui commented 1 year ago

The dc is now running on production. The first data should be available after lunch, because the scheduled job runs at 12:00 and 24:00 o'clock