Netatmo integration: API error: 500 (Internal Server Error)

rosch100 commented 3 months ago

The problem

Since the upgrade to 2024.6.1, the Netatmo integration stopped working. The error is:

Logger: homeassistant.components.netatmo
Quelle: components/netatmo/__init__.py:90
Integration: Netatmo (Dokumentation, Probleme)
Erstmals aufgetreten: 10:48:25 (7 Vorkommnisse)
Zuletzt protokolliert: 10:51:14

API error: 500 (Internal Server Error)

What version of Home Assistant Core has the issue?

Version core-2024.6.1 Installationstyp Home Assistant OS Entwicklung false Supervisor true Docker true Benutzer Virtuelle Umgebung false Python-Version 3.12.2 Betriebssystemfamilie Linux Betriebssystem-Version 6.6.28-haos-raspi CPU-Architektur aarch64 Zeitzone Europe/Berlin Konfigurationsverzeichnis /config

What was the last working version of Home Assistant Core?

2024.6.0

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Netatmo

Link to integration documentation on our website

https://www.home-assistant.io/integrations/netatmo/

Diagnostics information

home-assistant_netatmo_2024-06-09T08-54-46.855Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

I could (temporarily) resolve the issue by deleting and re-adding the integration. However, today the integration fails again with the same error. Deleting and Re-adding the integration is not a solution!

home-assistant[bot] commented 3 months ago

Hey there @cgtobi, mind taking a look at this issue as it has been labeled with an integration (netatmo) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `netatmo` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign netatmo` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

netatmo documentation netatmo source _{^{(message by IssueLinks)}}

cyhyryiys commented 3 months ago

Same problen for me. It seems to be connectef with the last Home Assistant Core update to 2024.6.1

Ph0B1uS commented 3 months ago

Same here, upgraded from 2024.3.3 to 2024.6.1 and the integration did not want to work afterwards. I'm using the docker version of HA.

I also tried downgrading to 2024.6.0 but that did not help unfortunately.

cyhyryiys commented 3 months ago

I found a Solution (?)

—>

Authentication problem solved as follows:

I called my own account at https://home.netatmo.com/ 3 Here is the menu My Account/Account Management/Third Party Apps There is a list of applications connected to the account Click into the “Home Assistant Cloud” app, select “Remove App” In Home assistant I brutally deleted the Netatmo integration I restarted HA I called wizzard “Add Integration” again, selected Netatmo On the Netatmo web page called by HA, I enabled the third party application

Ph0B1uS commented 3 months ago

I found a Solution (?)

—>

Authentication problem solved as follows:

I called my own account at https://home.netatmo.com/ 3 Here is the menu My Account/Account Management/Third Party Apps There is a list of applications connected to the account Click into the “Home Assistant Cloud” app, select “Remove App” In Home assistant I brutally deleted the Netatmo integration I restarted HA I called wizzard “Add Integration” again, selected Netatmo On the Netatmo web page called by HA, I enabled the third party application

This worked for me too but since it involved losing all historical data it's a less than ideal solution

Daniel-Joergensen commented 3 months ago

I'm unsure if this is related to newest HA version to be honest. It started suddenly for me, and I haven't updated to 2024.6.x yet, and run 2024.5.5.

If I delete the integration and re-add it, it will work again for anything between 2 to 12 hours.

Daniel-Joergensen commented 3 months ago

I wonder if this is related:

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

RemBrandNL commented 3 months ago

API error: 500 (Internal Server Error) 12:31:58 – (WARNING) Netatmo - message first occurred at June 9, 2024 at 09:31:50 and shows up 1203 times

Same issue on my side. Yes, did update to the latest version recently, but also hear that others have no issue having updated. Seems to support @Daniel-Joergensen’s comment.

nikee73 commented 3 months ago

Problem here also. I get this: Failed setup, will retry: 500, message='Internal Server Error', url=URL('https://account-link.nabucasa.com/refresh_token/netatmo')

kevlare commented 3 months ago

Same problem here. For me it coincided with switching from sd to ssd, but I don't think it's linked.

ghostwind66 commented 3 months ago

Same here. I have had the issue since last night

matteos1 commented 3 months ago

same problem

Friedi1970 commented 3 months ago

I wonder if this is related:

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

Hmm. If so, somebody here who knows how to deal with this? @cgtobi is that an issue which you need to solve?

eidoHA commented 3 months ago

Same problem here. Still in HA version 2024.5.3

ghostwind66 commented 3 months ago

I sorted out my issue by deleting my hub and re-adding it again. I noticed that some of the entities also hand slight name changes.

Daniel-Joergensen commented 3 months ago

I sorted out my issue by deleting my hub and re-adding it again. I noticed that some of the entities also hand slight name changes.

That has only helped for me, for a few hours, then it will stop working again

Friedi1970 commented 3 months ago

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

It won't last very long. It is due to the API changes of Netatmo (see https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370). The integration need to be fixed.

jeremy1515145 commented 3 months ago

Bonjour, même problème pour moi également lors d'une maj de HA.

NikolasRupp commented 3 months ago

I have the same Error

claudegraf commented 3 months ago

I have the same Problem since installation of

Core 2024.6.3 Supervisor 2024.06.0 Operating System 12.4 Frontend 20240610.1

Logger: homeassistant.components.netatmo Quelle: components/netatmo/init.py:90 Integration: Netatmo (Dokumentation, Probleme) Erstmals aufgetreten: 10:53:16 (7 Vorkommnisse) Zuletzt protokolliert: 10:57:19

API error: 500 (Internal Server Error)

ndrkxd commented 3 months ago

Same issue here.

Fotokroth commented 3 months ago

Update 6.1 caused other, even bigger, problems on my Green in addition to Netatmo. I went back to 6.0, deleted Netatmo and then set it up again. Now everything has been running stable for over 72 hours, now also with 6.3.

Lloydr149 commented 3 months ago

Same error here

RemBrandNL commented 3 months ago

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

sven-debug commented 3 months ago

I have removed my integration and-readded it again. ... Does mean all history is gone.

Same in my installation. Sadly, history values are gone now. But hey, it's just weather...

mariomaz87 commented 3 months ago

Same issue here.

floow77 commented 3 months ago

Same issue here

U2EZNeko commented 2 months ago

Same issue, worked one day, woke up to it not working at all.

I'd really like to avoid losing my historical data.

stguz commented 2 months ago

Hi, same hier since yesterday.

rosch100 commented 2 months ago

I found a Solution (?)

I could (temporarily) resolve the issue by deleting and re-adding the integration. However, today the integration fails again with the same error. Deleting and Re-adding the integration is NOT a permanent solution!

jeremy1515145 commented 2 months ago

Bonjour, pour ma part, en supprimant l'extension et en la réinstallant, j'ai seulement perdu les historiques mais l'intégration fonctionne à nouveau parfaitement depuis 2 semaines maintenant.

domibarton commented 2 months ago

Same error here, and I'm on 2024.6.4 :/

Please note I've been on 2023.12.xx before, and it already happened there. The update to 2024.6.4 didn't help.

domibarton commented 2 months ago

Are we sure this isn't related to this change?

When you refreshed an access_token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed

Starting from now, this behavior changed to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users

When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated

So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error

To fix it, you need to update the tokens value as soon as you get the newly generated ones

BurningSheep1 commented 2 months ago

Is there any solution already how to fix this? Can soneone explain in detail what to do? Thanks a lot :)

Are we sure this isn't related to this change?

When you refreshed an access_token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed Starting from now, this behavior changed to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error To fix it, you need to update the tokens value as soon as you get the newly generated ones

RemBrandNL commented 2 months ago

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

Update from my side. After re-adding it it has worked stable since then. Running on all latest versions.

Fotokroth commented 2 months ago

@RemBrandNL, does it still work after restarting the HA hardware?

NikolasRupp commented 2 months ago

@Fotokroth Yes. My HA restarts every das and it runs without Problems since 2 weeks

Fotokroth commented 2 months ago

@RemBrandNL, do you mean the system, i.e. the hardware or just the HA?

RemBrandNL commented 2 months ago

@RemBrandNL, do you mean the system, i.e. the hardware or just the HA?

Just did a full reboot of the hardware for you, still available.

domibarton commented 2 months ago

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

Update from my side. After re-adding it it has worked stable since then. Running on all latest versions.

means you lost all your history, right?

cgtobi commented 2 months ago

If you don't restart HA in between your history should not be cleared.

maehmann commented 2 months ago

Hey,

is this the recommended way? Or is there a possible solution with an update coming?

I‘m asking because I have seen entity ids change in the past …

thanks

Fotokroth commented 2 months ago

@RemBrandNL, Thank you. I just updated to 2024.7.1 and after the system restart, the Netatmo devices were still active on my device.

RemBrandNL commented 2 months ago

@maehmann, recommended, probably not. Took this route based on the posts of @cyhyryiys and @Ph0B1uS earlier in this thread. Main reason being that I don’t see ‘official’ acknowledgement of the issue and no fixes on the way if needed.

With re-adding my entities were messed up and couldn’t bother aligning them to keep my history. It’s “just” weather data and I don’t run automations that need historical data.

2 cents: I’m no dev, but there’s a chance that HA implementation is correct and the changes netatmo side just cause a one time fuckup. Seems to have to do with renewing the tokens to keep access to netatmo data. Integration misses a re-authentication on an existing connection for incidents like this it seems. If the cycle of getting a token with a proper renewal token the process will continue again as it’s supposed to do.

Cheers, Remco

maehmann commented 2 months ago

I removed the integration and readded it. That worked but when I chose Netatmo directly the authentication fails. Only the implementation through NabuCase worked. Does that mean that the communication now works like Netatmo <=> NabuCasa <=> My HA? If that is the case I wouldn't really like that, to be honest. I would rather want Netatmo <=> My HA.

cgtobi commented 2 months ago

No data is processed by NabuCasa. It's just that the cloud authentication is used.

Friedi1970 commented 2 months ago

@cgtobi do you plan to fix this issue? My integration is still not working and to be honest I don’t have a clue how to fix this on my own 😕

cgtobi commented 2 months ago

Of course, if only I'd have a clue what the issue would be. I cannot reproduce and the integration is utilising the authentication mechanism of HA.

Friedi1970 commented 2 months ago

Of course, if only I'd have a clue what the issue would be.

Isn’t it about this? https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

cgtobi commented 2 months ago

If that would be the case wouldn't everyone have to have the same issue?

home-assistant / core