Rudd-O / homeassistant-meteoswiss

:sun_behind_rain_cloud: :switzerland: Meteo Swiss Integration for Home Assistant (forked from websylv)
86 stars 10 forks source link

Error fetching meteoswiss data: Expecting value: line 1 column 1 (char 0) #28

Closed thor1872 closed 10 months ago

thor1872 commented 11 months ago

Logger: custom_components.meteoswiss Source: helpers/update_coordinator.py:322 Integration: Meteo Swiss (documentation, issues) First occurred: 02:01:04 (5 occurrences) Last logged: 02:14:04 Error fetching meteoswiss data: Expecting value: line 1 column 1 (char 0)

I all, I get this error after updating to the last version f2ad0cd with HACS and the entities are constantly becoming unavailable. I tried: uninstalling > restart HA > install it again > restart HA still same issue.

greeting thor1872 home-assistant_meteoswiss_2023-11-03T04-05-36.708Z.log

vistalba commented 11 months ago

Just came here to open exactly the same issue. For me it started yesterday evening around 17:40. This night around 4 it was working again until 4:22. Since then no weather available.

2023-11-03 08:18:40.081 DEBUG (MainThread) [custom_components.meteoswiss] Finished fetching meteoswiss data in 0.086 seconds (success: False)

Looks like an issue while fetching data from meteoswiss.

thor1872 commented 11 months ago

I just checked now that i got some sleep and your right looking at the history it started at 17:44 yesterday for me and got worse and worse, so maybe it's meteo swiss itself the problem and not the integration. And me updating yesterday was most likely just a coincidence.

Edit: i just re enabled the integration 20min ago and no issue for now.

vistalba commented 11 months ago

For me it still doesn‘t work at all. Same issue since 04:22 this night. Any Idea how to troubleshoot whats going wrong while fetching the data?

thor1872 commented 11 months ago

Weird it's doing fine for 2h now. Maybe try reinstalling the integration completely to see if it fix it.

vistalba commented 11 months ago

Weird it's doing fine for 2h now. Maybe try reinstalling the integration completely to see if it fix it.

Already tried this today morning without luck :(

thor1872 commented 11 months ago

Weird it's doing fine for 2h now. Maybe try reinstalling the integration completely to see if it fix it.

Already tried this today morning without luck :(

Then i am out of idea, the only thing i did was remove the integration at 5am before going to sleep and then put it back up at 15h and now it work. Maybe the station that you want to use or meteo swiss is still having issue, i really don't know sadly.

thor1872 commented 11 months ago

welp after ~3h without issue, it's doing it again. Screenshot_3395

vistalba commented 11 months ago

For me it still doesn't work. Just tried with a complete new install and config. No luck. So I tried to investigate whats wrong.

When I take a look to the code, I see various URLs. Some of them look like they do not exist anymore...


MS_BASE_URL = "https://www.meteosuisse.admin.ch"
JSON_FORECAST_URL = "https://app-prod-ws.meteoswiss-app.ch/v1/forecast?plz={}00&graph=false&warning=true"
MS_SEARCH_URL = "https://www.meteosuisse.admin.ch/home/actualite/infos.html?ort={}&pageIndex=0&tab=search_tab"
CURRENT_CONDITION_URL = (
    "https://data.geo.admin.ch/ch.meteoschweiz.messwerte-aktuell/VQHA80.csv"
)
STATION_URL = "https://data.geo.admin.ch/ch.meteoschweiz.messnetz-automatisch/ch.meteoschweiz.messnetz-automatisch_fr.csv"
STATION_TYPE_PRECIPITATION = "Précipitation"
STATION_TYPE_WEATHER = "Station météo"

MS_24FORECAST_URL = (
    "https://www.meteosuisse.admin.ch/product/output/forecast-chart/{}/fr/{}00.json"
)
MS_24FORECAST_REF = "https://www.meteosuisse.admin.ch//content/meteoswiss/fr/home.mobile.meteo-products--overview.html"

So I'm curious if we are really only two that running into issues or if there are several more users affected. Btw: Which station are you using? I use "KOP".

beatdoebeli commented 11 months ago

Same problem here. Just switched to the Rudd-O fork of Meteoswiss this morning, so I can't tell since when the problem exists.

vistalba commented 11 months ago

Same problem here. Just switched to the Rudd-O fork of Meteoswiss this morning, so I can't tell since when the problem exists.

Did you already try to switch back? Does it help?

badewanne1234 commented 11 months ago

Same here, it started around 6PM on 2-Nov, see screenshot. I did update to the latest version of the meteo integration today, but no luck.. grafik

vistalba commented 11 months ago

Same picture for me as @badewanne1234 posted. Sometimes it comes back for a few seconds then it stops working again.

thor1872 commented 11 months ago

For me it still doesn't work. Just tried with a complete new install and config. No luck. So I tried to investigate whats wrong.

When I take a look to the code, I see various URLs. Some of them look like they do not exist anymore...


MS_BASE_URL = "https://www.meteosuisse.admin.ch"
JSON_FORECAST_URL = "https://app-prod-ws.meteoswiss-app.ch/v1/forecast?plz={}00&graph=false&warning=true"
MS_SEARCH_URL = "https://www.meteosuisse.admin.ch/home/actualite/infos.html?ort={}&pageIndex=0&tab=search_tab"
CURRENT_CONDITION_URL = (
    "https://data.geo.admin.ch/ch.meteoschweiz.messwerte-aktuell/VQHA80.csv"
)
STATION_URL = "https://data.geo.admin.ch/ch.meteoschweiz.messnetz-automatisch/ch.meteoschweiz.messnetz-automatisch_fr.csv"
STATION_TYPE_PRECIPITATION = "Précipitation"
STATION_TYPE_WEATHER = "Station météo"

MS_24FORECAST_URL = (
    "https://www.meteosuisse.admin.ch/product/output/forecast-chart/{}/fr/{}00.json"
)
MS_24FORECAST_REF = "https://www.meteosuisse.admin.ch//content/meteoswiss/fr/home.mobile.meteo-products--overview.html"

So I'm curious if we are really only two that running into issues or if there are several more users affected. Btw: Which station are you using? I use "KOP".

I use PUY and like you say some link aren't working, MS_24FORECAST_URL and MS_24FORECAST_REF say "Page not found" so that might be a problem, but it's still weird that sometime it work and sometime not.

I just tried using the old weather.py file here

but the issue still happen, so it looks like it isn't the change made in the last update that's the problem. And the comment from badewanne1234 make me think that too.

So in the end i still don't know what's the issue. And i am not a dev so i can't really help much sadly.

Edit: i just updated Home assistant core from 2023.11.0 to 2023.11.1 and no change. And right now i am trying a update interval of 10min instead of 1min to see if maybe that change something, who know maybe meteo swiss don't like getting asked every minute but i doubt that it will change anything.

Edit2: changing to 10min interval didn't change anything, still getting the error after 8h...

gzxto1 commented 11 months ago

welp after ~3h without issue, it's doing it again. Screenshot_3395

Have exactly the same issue...

fredmatix commented 11 months ago

Same issue here. The main reason I was switching from the official meteo-swiss app to this fork (meteoswiss) was the connection issues. I do have even bigger gaps in the stats: grafik

Core 2023.11.1 Supervisor 2023.10.1 Operating System 11.1 Frontend 20231030.1

HACS: Integrations Version: 1.33.0 Frontend Version: 20220906112053

Meteo Swiss Version: latest

sereros commented 11 months ago

I‘m facing the same issue. It started at about on the 03.11. IMG_8736

kess78 commented 11 months ago

Same here... so happy to see that I'm not alone ;-)

thegoija commented 11 months ago

for some reason, get_forecast in hamsclientfork uses a xhtml header to receive json data. I have changed

_HEADERS = {
    "Accept": "text/html,application/xhtml+xml,application/xml;"
    "q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML"
    ", like Gecko) Chrome/1337 Safari/537.36",
}

to

_HEADERS = {
    "Accept": "text/html,application/json",
    "Accept-Encoding": "gzip, deflate, sdch",
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML"
    ", like Gecko) Chrome/1337 Safari/537.36",
}

directly in /usr/local/lib/python3.11/site-packages/hamsclientfork/client.py. Running now without issues for 24 hours.

image

JamborJan commented 11 months ago

Thanks for the effort @thegoija . Apparently it didn't change anything in my case. I'm still seeing the "Expecting value: line 1 column 1 (char 0)" error and no date is getting fetched. The last status change was 13 hours ago. In my case the issue started after updating to HA 2023.11. Also the update to 2023.11.1 didn't change anything. Like others replied already in this issues.

vistalba commented 11 months ago

Does not work for me :( Still the same issues. i guess it is not HA Version releated. The issue started for me on 3. November at around 17:44.

JamborJan commented 11 months ago

For me it's the same date where the issue started but it is also the same date I updated HA. I'll roll back HA later today and test that.

vistalba commented 11 months ago

For me it's the same date where the issue started but it is also the same date I updated HA. I'll roll back HA later today and test that.

For me I still was on 2023.10 when it stopped working. So I do not have a lot of hope that a rollback will help.

elduca commented 11 months ago

Same issue here after updating the integration

sebcou21 commented 11 months ago

Same behavior observerd on my side with issue starting on 02.11 around 17:40. I was still on 2023.10 at the time as well, so rollback will probably be useless.

TimMayle commented 11 months ago

I'm guessing the issue might be on meteoswiss API side. As we don't request the data of the same meteo station, we might could experience a different behaviour on our sides.

I couldn't find detailed information and status for this service. Best page found yet is this one: https://www.meteoswiss.admin.ch/weather/measurement-systems/data-management.html They might have changed their API without us knowing.

With more free time, I would create an account to test the API. Evidences yet are:

PS: As Frenck suggested, no need to add comments such as "same here". Just subscribe or follow the issue.

hb9eue commented 11 months ago

I wonder if this could be caused by some rate limiting on the API side?

vistalba commented 11 months ago

I wonder if this could be caused by some rate limiting on the API side?

Unlikely as for me the Meteo Swiss app, simple API Calls to the same endpoints and also download of the CSV File with the stations does work when I do this manually.

fredmatix commented 11 months ago

For me since 8. November 16:00 the issue disappeared: grafik Anyone else now having stable incoming data?

danielhad commented 11 months ago

Hi, yes. It seems to be stable now. At least in my system.

gzxto1 commented 11 months ago

the same for me.... :-)

On Fri, 10 Nov 2023 at 11:30, danielhad @.***> wrote:

Hi, yes. It seems to be stable now. At least in my system.

— Reply to this email directly, view it on GitHub https://github.com/Rudd-O/homeassistant-meteoswiss/issues/28#issuecomment-1805471698, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASCKH4QHNZO6ZWMXKYREWGDYDX64PAVCNFSM6AAAAAA63XMI22VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBVGQ3TCNRZHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Thomas Bürgi Dorfstr.37 8916 Jonen Switzerland

@.*** Home: +41(0)55 556 8788 Mobile: +41(0)79 359 0073

thor1872 commented 11 months ago

same here, working fine for two day now, so it was most likely a issue with meteo swiss itself. Screenshot_3402

Rudd-O commented 10 months ago

I'm thinking this is an issue with MS itself too. Enabling debug logging can show what the error condition was.