Problem with the connection to Netatmo API since 2023-12-04

[jostrasser]

Describe the bug Homebridge unable to start, plugin is preventing start because there are API issues with the Netatmo API.

[04/12/2023, 19:26:12] [eveatmo-platform] WARN - Netatmo: Error: getHealthyHomeCoachData error: Invalid access token
[04/12/2023, 19:26:29] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.
[04/12/2023, 19:26:49] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.
[04/12/2023, 19:27:09] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.
[04/12/2023, 19:27:29] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.
[04/12/2023, 19:27:49] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.

Version you use

• homebridge version: 1.7.0
• node version: 20.10.0
• plugin version: latest

To Reproduce Restart homebridge service

[jostrasser]

I think we are facing another API issue. Contacted Netatmo support and still waiting for feedback. I´ll keep you posted. BR/JO

[novski]

I have the same. The workaround here does not work on my side.

[mrmaximas]

anyone using api in two places at once like me? homebridge-eveatmo + node-red-contrib-netatmo-dashboard?

[mrmaximas]

I reset keys, deleted netatmo-token.json, generated a new access token with full r/w/a rights for all device types, put new client id, secret and refresh token into plugin settings and everything works again.

[jostrasser]

anyone using api in two places at once like me? homebridge-eveatmo + node-red-contrib-netatmo-dashboard?

Yes, I am also using both: node-red and homebridge-eveatmo. :) homebridge-eveatmo is working again after generating a new refresh token, node-red is broken and a fresh token does not solve the issue.

[jostrasser]

I reset keys, deleted netatmo-token.json, generated a new access token with full r/w/a rights for all device types, put new client id, secret and refresh token into plugin settings and everything works again.

Have you tested a restart of the bridge? Is it solved permanently after creating the new tokens? I created only a fresh access and refresh token... In my case a restart of the homebridge is breaking it again!

@skrollme I think there are adjustments required on the plugin if Netatmo changed their Auth method again... :(

[mrmaximas]

Have you tested a restart of the bridge? Is it solved permanently after creating the new tokens?

after full HB restart i have a problem again (((

[jostrasser]

Have you tested a restart of the bridge? Is it solved permanently after creating the new tokens?

after full HB restart i have a problem again (((

Yep, Is expected... :(

[jostrasser]

@skrollme

Info from Netatmo DEV Support:

We just did a modification on the token retrieval process :

When you refreshed an access_token using the associated endpoint [https://api.netatmo.com/oauth2/token, Netatmo servers](https://api.netatmo.com/oauth2/token,%C2%A0Netatmo%C2%A0servers) responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed

Starting from the 04/12/2023, this behavior changed to be compliant with the recommendations of the [RFC of the OAuth2 Authorization Framework](https://www.rfc-editor.org/rfc/rfc6749) (section 10.4) and improving the security of the data of our users
When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated

So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error
To fix it, you need to update the tokens value as soon as you get the newly generated ones.

[mrmaximas]

Yep, Is expected... :(

By the way i make additional "application" in dev.netatmo for NR.. Waiting for fix. In the new reality one token will not work in two systems as it will change after each request.

[fmarzocca]

In the new reality one token will not work in two systems as it will change after each request.

Unless the 2 systems don't share the same json file with the tokens

[mrmaximas]

Unless the 2 systems don't share the same json file with the tokens

they're two different systems.

[fmarzocca]

they're two different systems.

yes, but there are many ways to share a file between 2 different systems (provided they are using the same Netatmo account)

[mrmaximas]

many ways to share a file between 2 different systems

It won't work. Each time the api is accessed, the token will be renew, and the second system will get the expired token.

[fmarzocca]

You need to save the file at each request, and to keep the json file synchronized between the 2 systems

[mrmaximas]

You need to save the file at each request, and to keep the json file synchronized between the 2 systems

What's the point? I added a second app and have a couple of tokens that i can use at the frequency i need.

[fmarzocca]

What's the point? I added a second app and have a couple of tokens that i can use at the frequency i need.

Good, no point. But you said before you said that "one token will not work in two systems " and that's not true. Ok.

[mrmaximas]

But you said before you said that "one token will not work in two systems " and that's not true. Ok.

you are right, maybe it will work only in one case - when systems are unity and request data with the same frequency, but I have homebridge for homekit and node-red for export to influx then grafana.

[jostrasser]

Additional info from Netatmo:

The access_token must be refreshed every 3 hours (10800 seconds). This has always been the case
The difference was that if you refreshed your access_token before its expiration, refresh_token value always remained the same. If you wanted to refresh it but after expiration, the refresh_token value changed

Now, the 2 values change at each new /token request. So, you must catch the new generated refresh_token value each time a new request is done to the endpoint, and perform the next /token call with this new value

[Mindphazer73]

On my Netatmo app on my iPhone, I need to re-connect each time i use the app (I guess after the famous 3 hours) What a great idea from Netatmo, if they want their users to run away :-(

[mrmaximas]

On my Netatmo app on my iPhone, I need to re-connect each time i use the app (I guess after the famous 3 hours) What a great idea from Netatmo, if they want their users to run away :-(

I'm guessing it's not Netatmo's idea, but the very important Jedi Legrand engineers. :-)

[PvdGulik]

On my Netatmo app on my iPhone, I need to re-connect each time i use the app (I guess after the famous 3 hours) What a great idea from Netatmo, if they want their users to run away :-(

I don't recognize this till now...... Maybe later.....

[Mindphazer73]

I'm guessing it's not Netatmo's idea, but the very important Jedi Legrand engineers. :-)

Yeah you're probably right :-D

[jostrasser]

Another update from Netatmo. This is the reason why all the apps, plugins, integrations are back working again:

Indeed, teams had to do a rollback to fix a problem for multiple connections (if a 2nd device connects, it asks for a token and invalidates the access for the 1st connected device). I reported that it was a problem for third-party apps The plan is still to put in place this change. They are working on a solution, but I don't have for now more information about a release date

I requested detailed information when the next change will take place.

[Mindphazer73]

Well, it doesn't work for me I still have invalid token access

[Pajonk100]

Well, it doesn't work for me I still have invalid token access

same problem...

[jostrasser]

@Mindphazer73 & @Pajonk100

Please try the following: 1) Create a new Access and Refresh token with Postman 2) Manually enter both tokens into the netatmo-token.json file 3) Enter the same Refresh token into the config.json of Homebridge 4) Restart Homebridge service

This should actually work.

[PvdGulik]

I just created a new refresh token on dev.netatmo.com, and copied/pasted in Homebridge Eveatmo plugin in the settings, and voila, working again 😉 [07/12/2023, 18:29:53] [eveatmo platform] Initializing eveatmo platform... [07/12/2023, 18:29:53] [eveatmo platform] Authenticating using 'refresh_token' grant [07/12/2023, 18:29:57] [eveatmo platform] Loading new data from API for: weatherstation [07/12/2023, 18:29:57] Loading 3 accessories...

---

Van: Johannes Strasser @.> Verzonden: donderdag 7 december 2023 18:21 Aan: skrollme/homebridge-eveatmo @.> CC: Tebogo @.>; Comment @.> Onderwerp: Re: [skrollme/homebridge-eveatmo] Problem with the connection to Netatmo API since 2023-12-04 (Issue #77)

@Mindphazer73https://github.com/Mindphazer73 & @Pajonk100https://github.com/Pajonk100

Please try the following:

1. Create a new Access and Refresh token with Postman
2. Manually enter both tokens into the netatmo-token.json file
3. Enter the same Refresh token into the config.json of Homebridge
4. Restart Homebridge service

This should actually work.

— Reply to this email directly, view it on GitHubhttps://github.com/skrollme/homebridge-eveatmo/issues/77#issuecomment-1845772432, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACH4FIROVQOQZEQHRT562LDYIH3K7AVCNFSM6AAAAABAGOL7BOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBVG43TENBTGI. You are receiving this because you commented.Message ID: @.***>

[Pajonk100]

@Mindphazer73 & @Pajonk100

Please try the following:

1. Create a new Access and Refresh token with Postman
2. Manually enter both tokens into the netatmo-token.json file
3. Enter the same Refresh token into the config.json of Homebridge
4. Restart Homebridge service

This should actually work.

problem was in netatmo-token.json - old tokens before my testing what was wrong... Now its working :) thx :D

[mrmaximas]

@Mindphazer73 & @Pajonk100

Please try the following:

1. Create a new Access and Refresh token with Postman
2. Manually enter both tokens into the netatmo-token.json file
3. Enter the same Refresh token into the config.json of Homebridge
4. Restart Homebridge service

This should actually work.

all Hombridge or plugin Bridge only?

[mrmaximas]

@Mindphazer73 & @Pajonk100 Please try the following:

1. Create a new Access and Refresh token with Postman
2. Manually enter both tokens into the netatmo-token.json file
3. Enter the same Refresh token into the config.json of Homebridge
4. Restart Homebridge service

This should actually work.

problem was in netatmo-token.json - old tokens before my testing what was wrong... Now its working :) thx :D

In my case it was always enough to delete netatmo-token.json and put a new refresh token in plugin config

[Mindphazer73]

Thanks I just deleted netatmo-token.json, and restarted Homebridge, and it works again

[jostrasser]

@mrmaximas

all Hombridge or plugin Bridge only?

Only for the eveatmo plugin and a restart of the Homebridge service.

In my case it was always enough to delete netatmo-token.json and put a new refresh token in plugin config

Should also work if the plugin is able to create it again. I ran into an issue in the past, that's the reason why I suggested to replace the keys. ;)

I hope @skrollme can create a fix until Netatmo redo the change on their platform.

[mrmaximas]

Only for the eveatmo plugin and a restart of the Homebridge service.

A couple of days ago it stopped working after a full restart of homebridge. Since I'm already tired of generating tokens, I'm not touching or restarting anything yet.

[jostrasser]

A couple of days ago it stopped working after a full restart of homebridge. Since I'm already tired of generating tokens, I'm not touching or restarting anything yet.

Yes, I had the same behavior after a Homebridge restart. If the bridge is not restarted, everything should continue to run, also if Netatmo will redo the change.

The sticking point was the creation of new keys when the bridge restarts I think. So the goal should be to make it persistent again.

[skrollme]

Is this still an open case, @jostrasser ?

[jostrasser]

Is this still an open case, @jostrasser ?

Hey @skrollme , Right now there is no issue because they rolled back the Auth method again. I think this was related to it. Netatmo confirmed me "issues with 3rd party apps" and confirmed this as reason for the rollback.

They have planned to enable the new auth method again (May 29th 2024) but I don't know if the current implantation is working afterwards. I hope so ;)

General: If the API is inaccessible the plugin won't start / is hanging at "Starting".

It would be cool if you can add a timeout with a following retry (every 5mins) until the API comes back online. This would keep Homebridge up and running even if the API is not working correctly.

Thanks!

---

ADD: INFOMAIL FROM NETATMO

Today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, the Netatmo servers respond with a pair of tokens: an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned access token is identical but has an expiration time extended by 3 hours.

The Refresh Token is not renewed.

Starting May 29, 2024, this behavior will change to comply with the OAuth2 Authorization Framework RFC recommendations (section 10.4) and improve the security of our users' data.

When refreshing the tokens, the Access Token and Refresh Token will be different from the previous ones and the old tokens will be invalidated.

What does this mean for you?

If you already store the tokens returned by the API when refreshing your tokens, this change will not affect you.

If you do not update the tokens when refreshing them, your users will be logged out because the old tokens will be invalidated.

If for a given user, you have stored the same token in different places (for example: an application and its widget), you must have a way to synchronize them because the new token will be applied for the entire application.

[skrollme]

General: If the API is inaccessible the plugin won't start / is hanging at "Starting".

Correct, I think this is the main problem here.

It would be cool if you can add a timeout with a following retry (every 5mins) until the API comes back online. This would keep Homebridge up and running even if the API is not working correctly.

I'm not sure if the plugin's overall architecture supports this kind of approach. This would also be tricky to stage this environment/situation for testing purposes 🤔

[jostrasser]

I'm not sure if the plugin's overall architecture supports this kind of approach. This would also be tricky to stage this environment/situation for testing purposes 🤔

Understood. If there is no way to workaround a hanging startup of Homebridge if the API is down we only can disable the plugin until the API is back to get Homebridge back online (or we have to wait for the API).

For me it's okay because I am backing up my HomeKit telemetry and can easily restore it even I loose the room assignments of the sensors. (can be done with the "Controller for HomeKit" app)

If you are planning to overwork the plugin: please keep this in mind as a feature request please 😉

Thank you!