search

cinemast / covid19-at

COVID-19 Stats and Maps for Austria
https://covid19.spiessknafl.at
BSD 3-Clause "New" or "Revised" License
21 stars 2 forks source link

data source issue with Lat/Lng and province #2

Closed jostrasser closed 4 years ago

jostrasser commented 4 years ago

Hi, thanks for implementing the Lat/Long data in your data source, but I am having some issues with it. I am almost done with a dashboard which I can share with you to implement it in the next build.

I am now trying to implement some maps, but I have troubles with to get the lat/long fields, I doesn't get them correctly tagged on the world map.

example: cov19_world_infected{country="Austria",continent="Europe",latitude="47.516231",longitude="14.550072"} 1016

Query: metrics - cov19_world_infected > 0

Error on world map: Invalid LatLng object: (undefined, undefined)

I am getting back the following if I take a look to the data panel: Time name continent country instance job Value

I am missing the province, latitude and longitude. This error can occur because some records are not delivering all fields.

Any idea how I can solve this?

Thanks JO!

cinemast commented 4 years ago

From here I can see that the austria deaths statistics lack lat/long attributes. But you are querying a different time series.

Can you share a dashboard json export to reproduce this?

cinemast commented 4 years ago

Please check if https://github.com/cinemast/covid19-at/commit/fe58de1b88fe0565b8c65b70f1e96057f92a129e fixes the problem

jostrasser commented 4 years ago

I will try it. In the world map settings under “map data options” I have selected location data “table”.

I will create a sample dashboard and share it with you.

Thanks!

jostrasser commented 4 years ago

Hi again, looks good for now.

I will finish the dashboard tomorrow. How should I share it with you?

BR/JO!

cinemast commented 4 years ago

A gist would be fine. Looking forward to it.

jostrasser commented 4 years ago

Hi, all done. :) I have made two dashboards (INIT release): GER and ENG version.

You can find the json files here: https://gist.github.com/jostrasser/4b21babc89987b33c3091b98ef2ce11d

Sample screenshots: https://snipboard.io/rCqgmf.jpg https://snipboard.io/FZHdYM.jpg https://snipboard.io/1tXzGO.jpg

I will have some fine tunings the next few days and keep you posted. I can see some breaks in different stats / lines. They are related to some changes on the backend (prometheus, coordinates,.... )... If you want to fix it you have to do this on the database side.

BR/JO!

cinemast commented 4 years ago

This looks really great! Thanks for sharing, I will include some of it.

jostrasser commented 4 years ago

This looks really great! Thanks for sharing, I will include some of it.

Thanks ;) You can use the full dashboards if you want, I will inform you when I update something on it (if you tell me, how) ;)

BR/JO!

jostrasser commented 4 years ago

I have added now the Infection Rate for AT, updated Gist.

BR/JO!

cinemast commented 4 years ago

Thanks! I included it in https://github.com/cinemast/covid19-at/commit/435598578faf8980a47185efb5b68e0dc7c97c8e

It also contains fixes for the duplicate data (you need max(<metric>) without(instance,job,latitude,longitude).

cinemast commented 4 years ago

Feel free to open a PR if you want it to be updated. It is available here: https://covid19.spiessknafl.at/d/eS-fSwXWz/covid-19-global-overview?orgId=1&refresh=1m

jostrasser commented 4 years ago

Great! I have changed the behavior for mobile devices (mouse wheel issue). For now all looks fine with REV3 on my Gist... I keep you posted... :) Thanks!

BR/JO!

jostrasser commented 4 years ago

...and Revision 4 now merges your fix with "max()..." and my changes in REV3 (Infection Ratio for Austria). I think this version is the best for now ;)

Thanks JO!

jostrasser commented 4 years ago

Another info... sometimes I can see "No Data" in some "Stats Fields". Right now we have no data in "Deaths in Austria".

Could you check this on your side please? Thanks JO!

cinemast commented 4 years ago

I just saw it, yes I am on it.

https://www.sozialministerium.at/Informationen-zum-Coronavirus/Neuartiges-Coronavirus-(2019-nCov).html removed the details for deaths. I will at the summarized value though.

cinemast commented 4 years ago

ECDC data is also not very good at the moment. I tried to make the best out of it by normalizing the country names in a defined way.

The summarized deaths in austria value is also defined now: cov19_dead

jostrasser commented 4 years ago

I think the Sozialministerium will bring the deaths back. Have seen this sometimes the last few days. It would be a cool workaround to keep the last value and overwrite it if a newer death value is coming. Is that possible on the database side?

jostrasser commented 4 years ago

Do you think I only should switch on the Stats query from "sum(cov19_detail_dead{country="Austria"})" to "cov19_dead" ?

Is there a way to get the province in Austria for deaths?

jostrasser commented 4 years ago

Ok, there is maybe an official one with much more details.

See

https://info.gesundheitsministerium.at/d/87BA8__Wk1/covid-19-offentlich-zugangliche-information?orgId=1&refresh=5m

BR/JO!

cinemast commented 4 years ago

Thanks! I just saw. I guess we are out of business :D Thanks for your help.

jostrasser commented 4 years ago

Do we get this data also, or is there no way? Thanks

jostrasser commented 4 years ago

On the other side U‘re right. ;)

Do you know, is there a way to „link“ dashboard into my own Grafana instance?

Do you plan to shutdown your Prometheus DB for the dashboard?

Thanks!

BR/JO!

cinemast commented 4 years ago

It appears via the grafana datasource API it is possible:

curl 'https://info.gesundheitsministerium.at/api/tsdb/query' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:74.0) Gecko/20100101 Firefox/74.0' -H 'Accept: application/json, text/plain, */*' --compressed -H 'X-Grafana-Org-Id: 1' -H 'Content-Type: application/json;charset=utf-8' -H 'X-TS-AJAX-Request: true' -H 'Origin: https://info.gesundheitsministerium.at' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Referer: https://info.gesundheitsministerium.at/d/87BA8__Wk1/covid-19-offentlich-zugangliche-information?orgId=1&refresh=5m'  --data '{"from":"1582498800000","to":"1584557020522","queries":[{"refId":"A","intervalMs":10800000,"maxDataPoints":148,"datasourceId":3,"rawSql":"select Bezirk, count(*) as Anzahl from GRAFANA_Covid_Faelle group by Bezirk order by Bezirk","format":"table"}]}'
cinemast commented 4 years ago

No, I will keep it running for now, since I still enjoy the project.

jostrasser commented 4 years ago

Great! I love the combination Austria / Europe and worldwide. A really good overview.

If we can get also the details which they are using will be great. Any ideas to get them into the database?