search

noi-techpark / bdp-core

Open Data Hub / Timeseries Core
https://opendatahub.com
Other
9 stars 4 forks source link

EPIC: As an Open Data Hub manager I would like that Grafana can go beyond the monitoring of the functioning of the Data Collectors, and also check if the single stations are working and (if possible) if all their connected sensors are working #267

Closed rcavaliere closed 1 year ago

rcavaliere commented 1 year ago

https://grafana.testingmachine.eu/d/NQ9asNaVk/missing-data-by-station?from=now-6M&to=now&orgId=1

dulvui commented 1 year ago

I created now a first draft of a simple table dashboard that shows all stations that didn't get data since the last 24 hours, with the datatype, station id and origin: https://grafana.testingmachine.eu/d/NQ9asNaVk/missing-data-by-station?from=now-6M&to=now&orgId=1

Lets see together if this view can make sense or if another view with other data is more useful.

rcavaliere commented 1 year ago

@dulvui looks good! I think we need some more metadata related to the station, to be able to filter (e.g. name). Also the reference to the stationtype would be important to have.

dulvui commented 1 year ago

@rcavaliere Awesome! I added now station name, station type. I could also include the complete metadata of every station if needed. Basically every data than you can see in the database, can be included in this dashboard.

rcavaliere commented 1 year ago

@dulvui that's really what I need, very good! One question: in this table we show the status only of stations / sensors that have issues or all stations / sensors are present?

dulvui commented 1 year ago

@rcavaliere This table is showing all stations/sensors that had no data since 24 hours or more. So technically some could be in this list, but actually don't have a problem, because the interval of the cron job to get the data runs every 24 hours or more. I'll try to add the delta time of the interval between measurements to the table, so we could see real problems.

rcavaliere commented 1 year ago

@dulvui another field that would be useful: the period. In some cases we have multiple measurements for the same data type, but with different periods. It is important to have also this field to filter out the different type of measurements available

dulvui commented 1 year ago

@rcavaliere I added now period to the dashboard. I also tried to add the delta time, as I mentioned in my previous comment, but since for that I would have to do queries on the measurementhistory table, it would become very slow to load etc. I could imagine that if @clezag takes a look at the dashboard, he might know how to create a fast query, for getting the delta time of the latest measurements. Lets see together on Monday in the office.