jdemaeyer / brightsky

JSON API for DWD's open weather data.
https://brightsky.dev/
MIT License
299 stars 18 forks source link

Investigate faulty MOSMIX parsing #102

Closed jdemaeyer closed 1 year ago

jdemaeyer commented 4 years ago

When unzipped, each station in the MOSMIX file looks roughly like this:

<Placemark>
    <name>10315</name>
    <description>MUENSTER/OSNABR.</description>
    <ExtendedData>
        <Forecast elementName="TTT">
            <value>      282.95     283.15     283.35     283.45     283.45     283.15     282.65     282.25     281.95     281.85     281.85     281.55     281.05     280.85     280.65     280.55     280.35     280.15     280.25     280.15     280.35     280.55     281.15     281.75     282.25     282.45     283.05     283.25     283.35     282.85     282.35     281.95     281.65     281.15     280.95     280.75     280.55     280.65     280.65     280.65     280.55     280.45     280.45     280.25     280.35     280.55     281.05     281.95     282.75     283.55     283.85     284.25     284.05     283.65     283.15     282.45     281.75     280.85     280.05     279.95     279.55     279.15     279.05     278.55     278.45     278.35     278.25     278.05     278.25     279.05     280.15     281.45     282.55     283.45     283.95     284.25     284.45     283.85     283.25     282.45     281.55     280.85     280.15     279.65     279.55     279.15     279.05     278.95     278.85     278.55     278.85     278.85     279.15     279.55     280.25     281.25     282.15     282.85     283.45     283.85     283.85     283.45     283.05     282.45     282.05     281.55     281.35     281.25     281.05     280.75     280.25     280.05     279.95     279.85     279.75     279.85     279.85     280.45     281.25     282.05     282.75     283.45     283.85     284.25     284.25     284.05     283.45     282.85     282.15     281.65     281.35     281.15     280.75     280.45     279.75     279.85     279.65     279.35     279.25     279.35     279.45     279.75     280.75     281.95     283.35     284.45     285.15     285.65     285.75     285.45     284.95     284.25     283.55     282.65     282.35     282.25     281.95     281.85     281.65     281.45     281.25     281.25     281.15     281.25     281.45     281.95     282.75     283.75     284.85     285.85     286.35     286.55     286.75     286.45     285.95     285.45     284.65     284.35     283.95     283.65     283.35     283.35     283.05     282.85     282.85     282.75     282.75     282.75     282.85     283.15     283.85     284.75     285.45     286.15     286.75     287.05     287.25     286.95     286.45     285.75     285.05     284.85     284.65     284.45     284.25     284.15     284.25     284.15     283.95     283.85     283.85     283.85     283.85     284.15     284.65     285.55     286.35     287.15     287.95     288.35     288.45     288.15     287.55     286.45     283.65     283.55     283.55     283.45     283.35     283.25     283.05     282.95     282.85     282.75     282.75     280.85     280.95     281.25     281.85     282.75</value>
        </Forecast>
    </ExtendedData>
    <Point>
        <coordinates>7.7,52.13,48.0</coordinates>
    </Point>
</Placemark>

Since 10 UTC on Oct 13th, something about the file changed that makes parsel no longer recognize the very last Point node of the XML. It does, however, still recognize the last ExtendedData node - so we have station id, name, and weather metrics, but no station coordinates.

At the time I incorrectly added e8a6a41 because I thought the issue was on the DWD side, but after manual inspection it turns out that the file looks absolutely normal and that parsel does pick up the Point node when I move the whole Placemark node into a separate file. So maybe it's related to how we handle compression or file encoding?

The affected station is K626 (HERMESKEIL).