Zamg 11077 Station not working?

LuisThe0ne commented 2 years ago

The problem

When I configure Zamg to use the Station with ID 11077 (Grabbed the ID from link), also tried using the id from here, but both didn't work for me, any tips or ideas how to fix that?

What version of Home Assistant Core has the issue?

2022.8.6

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Zamg

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zamg/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Exact Error: Configured ZAMG station_id (11077) is not a known station

probot-home-assistant[bot] commented 2 years ago

zamg documentation zamg source _{^{(message by IssueLinks)}}

lcnittl commented 2 years ago

The ZAMG integration page states:

Only observations for capital cities are publicly available. You can check the list of stations in CSV format.

Brunn am Gebirge is, unfortunately, not among the listed - supported - stations, so I guess it is not possible to use this station with this integration.

LuisThe0ne commented 2 years ago

Any Ideas why? Atleast in the ZAMG Download Portal the Data is availible.

lcnittl commented 2 years ago

Digging a bit into this topic, I am wondering, whether the ZAMG integration can be inproved by using different data/a different API.

The current API ( http://www.zamg.ac.at/ogd/ ) seems to be rather limited in the number of stations.

Checking the ZAMG Datahub (don't aks how I got there), after entering any of the datasets, and clicking on "API Dokumentation → Gehe zum Datensatz", one reaches an API documentation (for a different API than ogd) at: https://dataset.api.hub.zamg.ac.at/v1/docs/?anonymous=true

Now, probing the API a bit:

GET https://dataset.api.hub.zamg.ac.at/v1

{
    "detail": "To view available datasets, use the /datasets URI.",
    "url_schema": "/datasets?type=<EndpointType>&mode=<EndpointMode>",
    "examples": [
        "https://dataset.api.hub.zamg.ac.at/v1/datasets",
        "https://dataset.api.hub.zamg.ac.at/v1/datasets?type=grid",
        "https://dataset.api.hub.zamg.ac.at/v1/datasets?mode=current",
        "https://dataset.api.hub.zamg.ac.at/v1/datasets?type=timeseries&mode=historical"
    ],
    "types": [
        "grid",
        "timeseries",
        "station"
    ],
    "modes": [
        "historical",
        "current"
    ]
}

One can easily get a list of endpoints that serve current data:

GET https://dataset.api.hub.zamg.ac.at/v1/datasets?mode=current

{
    "/station/current/tawes-v1-10min": {
        "type": "station",
        "mode": "current",
        "response_formats": [
            "geojson"
        ]
    }
}

As detailed in the API documentation, the available metadata (measrement parameter, station_ids, and others) for this endpoint, can be obtained via its /metadata endpoint (output truncated here):

GET https://dataset.api.hub.zamg.ac.at/v1/station/current/tawes-v1-10min/metadata

{
    "title": "TAWES",
    "parameters": [
        {
            "name": "DD",
            "long_name": "Windrichtung",
            "desc": "Windrichtung der letzten 10 Minuten",
            "unit": "°"
        },
                // removed 49 entries for a shorter comment
        {
            "name": "ZEITX",
            "long_name": "Zeitpunkt der Windspitze",
            "desc": "Zeitpunkt der Windspitze der letzten 10 Minuten im Format hhmm",
            "unit": ""
        }
    ],
    "frequency": "10T",
    "type": "station",
    "mode": "current",
    "response_formats": [
        "geojson"
    ],
    "time": "2022-08-23T14:30:00+00:00",
    "stations": [
        {
            "type": "INDIVIDUAL",
            "id": "11266",
            "group_id": null,
            "name": "ACHENKIRCH CAMPINGPLATZ",
            "state": "Tirol",
            "lat": 47.49944444444444,
            "lon": 11.706666666666665,
            "altitude": 931,
            "valid_from": "2016-06-28T00:00:00+00:00",
            "valid_to": "2023-08-23T14:31:52+00:00",
            "has_sunshine": false,
            "has_global_radiation": false
        },
                // removed 226 entries for a shorter comment
        {
            "type": "INDIVIDUAL",
            "id": "11333",
            "group_id": null,
            "name": "HOHENAU",
            "state": "Niederösterreich",
            "lat": 48.60583333333334,
            "lon": 16.913055555555555,
            "altitude": 151,
            "valid_from": "2007-03-30T00:00:00+00:00",
            "valid_to": "2023-08-23T14:31:52+00:00",
            "has_sunshine": false,
            "has_global_radiation": false
        }
    ]
}

and indeed, Brunn am Gebirge (id=11077) is one of them:

        {
            "type": "INDIVIDUAL",
            "id": "11077",
            "group_id": null,
            "name": "BRUNN AM GEBIRGE",
            "state": "Niederösterreich",
            "lat": 48.106944444444444,
            "lon": 16.27,
            "altitude": 291,
            "valid_from": "2008-08-27T00:00:00+00:00",
            "valid_to": "2023-08-23T14:31:52+00:00",
            "has_sunshine": false,
            "has_global_radiation": false
        },

For Brunn am Gebirge, the current data ("2022-08-23T14:50:00+00:00") – I selected a few parameters – is:

GET https://dataset.api.hub.zamg.ac.at/v1/station/current/tawes-v1-10min?parameters=DDX,FFAM,FFX,P,RRM,TL,TLAM&station_ids=11077

{
    "media_type": "application/json",
    "type": "FeatureCollection",
    "version": "v1",
    "timestamps": [
        "2022-08-23T14:50:00+00:00"
    ],
    "features": [
        {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [
                    16.27,
                    48.106944444444444
                ]
            },
            "properties": {
                "parameters": {
                    "DDX": {
                        "name": "Windrichtung der Windspitze",
                        "unit": "°",
                        "data": [
                            282
                        ]
                    },
                    "FFAM": {
                        "name": "Arithmetisches Mittel der Windgeschwindigkeit",
                        "unit": "m/s",
                        "data": [
                            3.4
                        ]
                    },
                    "FFX": {
                        "name": "Windspitze",
                        "unit": "m/s",
                        "data": [
                            7.4
                        ]
                    },
                    "P": {
                        "name": "Luftdruck",
                        "unit": "hPa",
                        "data": [
                            982.7
                        ]
                    },
                    "RRM": {
                        "name": "Niederschlagsmelder der letzten 10 Minuten",
                        "unit": "min",
                        "data": [
                            0
                        ]
                    },
                    "TL": {
                        "name": "Lufttemperatur",
                        "unit": "°C",
                        "data": [
                            18.7
                        ]
                    },
                    "TLAM": {
                        "name": "Lufttempertur arithmetisches Mittel",
                        "unit": "°C",
                        "data": [
                            18.8
                        ]
                    }
                },
                "station": "11077"
            }
        }
    ]
}

Summing up

228 stations is much more then the 21 stations available via ogd (which, by the filename of the csv file that is linked in the current ZAMG integration is "TAWES 1h" data).

This API furthermore allows to get data for multiple stations with a single request (station_ids), and allows to filter the desired data (parameters) via query strings.

lcnittl commented 2 years ago

Ahh - so the current "API" (odg) apparently merely returns the csv file that is -- according to ZAMG integration documentation -- used to get the station ID.

GET http://www.zamg.ac.at/ogd/
user-agent: httpyac
accept: */*
accept-encoding: gzip, deflate, br

HTTP/1.1  200  OK
date:                         Tue, 23 Aug 2022 14:58:55 GMT
server:                       Apache
access-control-allow-methods: GET
access-control-allow-origin:  *
content-disposition:          attachment; filename="tawes1h.csv"
content-length:               2029
expires:                      Sat, 26 Jul 1997 05:00:00 GMT
cache-control:                no-cache, must-revalidate
pragma:                       no-cache
x-genhost:                    www14
connection:                   close
content-type:                 text/csv

"Station";"Name";"Höhe m";"Datum";"Zeit";"T °C";"TP °C";"RF %";"WR °";"WG km/h";"WSR °";"WSG km/h";"N l/m²";"LDred hPa";"LDstat hPa";"SO %"
11010;"Linz/Hörsching";298;"23-08-2022";"16:00";17,2;15,6;90;310;9,4;;16,6;0;1018,2;981,7;0
11012;"Kremsmünster";383;"23-08-2022";"16:00";16,8;15;89;55;9;83;16,2;0;1018,4;973,1;0
11022;"Retz";320;"23-08-2022";"16:00";18,6;17,1;90;288;9,4;275;19,4;0;1015,8;978,8;0
11035;"Wien/Hohe Warte";203;"23-08-2022";"16:00";19,3;15;77;286;21,2;293;48,6;0;1016;991,8;49
11036;"Wien/Schwechat";183;"23-08-2022";"16:00";20;14,5;71;330;24,1;;38,9;0;1016,3;995;14
11101;"Bregenz";424;"23-08-2022";"16:00";26,7;9,7;35;312;17,3;332;29,5;0;1014,4;965,6;100
11121;"Innsbruck";579;"23-08-2022";"16:00";26;6,1;28;96;11,2;64;23;0;1013,9;949;100
11126;"Patscherkofel";2247;"23-08-2022";"16:00";10,5;3,9;64;17;13;45;24,5;0;;780,3;100
11130;"Kufstein";495;"23-08-2022";"16:00";24,5;10,5;41;51;11,2;50;27,4;0;1015,2;960;100
11150;"Salzburg";430;"23-08-2022";"16:00";25;9;36;300;13;;25,9;0;1015,6;965,6;100
11155;"Feuerkogel";1618;"23-08-2022";"16:00";11,9;5,2;64;314;20,2;307;34,6;0;;841,2;19
11157;"Aigen im Ennstal";640;"23-08-2022";"16:00";20,4;9,9;51;219;18,4;214;40,3;0;1015,9;942,5;29
11171;"Mariazell";866;"23-08-2022";"16:00";13,6;11,7;88;253;7,6;201;15,5;0;1018,1;919,8;0
11190;"Eisenstadt";184;"23-08-2022";"16:00";21,2;14,5;66;323;21,6;310;45;0;1014,8;993,8;22
11204;"Lienz";659;"23-08-2022";"16:00";27;3,1;21;93;16,9;69;43,9;0;1011;937,7;78
11240;"Graz/Flughafen";340;"23-08-2022";"16:00";21,3;12,2;56;340;16,6;;25,9;0;1014,9;973,4;0
11244;"Bad Gleichenberg";280;"23-08-2022";"16:00";22,6;14,8;61;311;10,8;331;19,4;0;1014,8;983,8;88
11265;"Villacher Alpe";2140;"23-08-2022";"16:00";11,4;6,3;70;343;26,3;345;31;0;;787,7;68
11331;"Klagenfurt/Flughafen";447;"23-08-2022";"16:00";22,9;11,8;50;128;4,7;129;15,8;0;1014,3;963,3;3
11343;"Sonnblick";3105;"23-08-2022";"16:00";1,2;0,9;98;41;25,2;43;37,1;;;701,4;11
11389;"St. Pölten";270;"23-08-2022";"16:00";16,6;15,3;92;213;17,6;222;30,6;0;1017,5;986;0

LuisThe0ne commented 2 years ago

Interesting, I was struggeling with the same new and "interesting designed" Api for half an hour now because some links listed here didn't even work and the samples listed here didn't too. Not sure what or if I did anything wrong but ok. I think the best way to obtain the station Ids is the csv here. I saw that u used GET https://dataset.api.hub.zamg.ac.at/v1/station/current/tawes-v1-10min?parameters=DDX,FFAM,FFX,P,RRM,TL,TLAM&station_ids=11077 for the request which seems interesting because the DataHub where the Api is located uses other Ids. So creating a new Implementation with the new Api would be the best. :)

lcnittl commented 2 years ago

because some links listed here didn't even work and the samples listed here didn't too. Not sure what or if I did anything wrong but ok.

Not sure if that the case, but I found that using current with any resource_id other than tawes-v1-10min fails (redirects to a login page). This is where the filtering for mode=current helped me to find that tawes-v1-10min is indeed the only resource_id providing current data.

LuisThe0ne commented 2 years ago

Well, I think tawes is really the only option to fetch data, as described on the zamg homepage tawes is the "codename" for a "teilautomatische Wetterstation". So probably the other datasets are for other data.

LuisThe0ne commented 2 years ago

I quickly (1 hour later...) wrote a code example on how the Api could be used.

import requests
import json

print("Enter Zamg Station to get Info from")

selected_station = input()

tawesmetadata = json.loads(requests.get('https://dataset.api.hub.zamg.ac.at/v1/station/current/tawes-v1-10min/metadata').content)

tawesstations = tawesmetadata["stations"]

stationids = []

for item in tawesstations:
    stationids.append(item["id"])

if selected_station in stationids:
    print("Your Station exists")

    requesturl = "https://dataset.api.hub.zamg.ac.at/v1/station/current/tawes-v1-10min?parameters=TL,P,FFAM&station_ids=" + selected_station

    stationweather = json.loads(requests.get(requesturl).content)["features"][0]["properties"]["parameters"]

    stationtemperatur = str(stationweather["TL"]["data"][0])
    stationtemperaturunit = stationweather["TL"]["unit"]

    stationpressure = str(stationweather["P"]["data"][0])
    stationpressureunit = stationweather["P"]["unit"]

    stationwind = str(stationweather["FFAM"]["data"][0])
    stationwindunit = stationweather["FFAM"]["unit"]

    print("Temperatur: " + stationtemperatur + stationtemperaturunit + ",Air Pressure: " + stationpressure + stationpressureunit + ",Windspeed: " + stationwind + stationwindunit)
else:

    print("Can't find you Station")

If the code is terrible I am really sorry, my python experience is = 0, so the code mainly contains stackoveflow and google snippets. With all of that said I still hope it is useful to someone. :smiley:

PS: I would really like to implement the Integration in homeassistant myself but I don't understand how I could do that, so any help is appreciated.

LuisThe0ne commented 2 years ago

Now I wrote a simple cli client for the API (https://github.com/LuisThe0ne/zamg-api-cli-client). I hope that makes it easier to implement into homeassistant, as I have been struggeling with it for several hours now :(

lcnittl commented 2 years ago

Thanks for all your effort. I think it is best to start from the existing integration's code, and adjust it to read the data from the returned GeoJSON. In the meantime, it might be simplest to just use the RESTful integration (I wish my PR was already merged, as it - so do I think - helps in understanding the usage of the RESTful integration). With this it is rather easy to read the data from the new API (no immediate need for a new ZAMG integration).

killer0071234 commented 2 years ago

Hey, There is already a PR #66469 waiting to be merged with the change to use the newAPI. As mentioned by @LuisThe0ne

killer0071234 commented 1 year ago

The zamg integration included in release 2022.11 can now handle station 11077. In my opinion this issue can now be closed.

LuisThe0ne commented 1 year ago

@killer0071234 Thanks for implementing that! I will close the issue now.

home-assistant / core