New authentification method

[julien94800]

Hello, Starting October, current way to connect to Netatmo server will be removed and a new one introduced. We will need to find an other way to connect and keep this plugin working

[julien94800]

To improve the security of our products, we inform you that the Client Credentials grant type method will be completely removed. It will no longer be possible to authenticate with the username and password of the user.

The effective date of this update is October 2022.

How can you authenticate with Netatmo API ? From this date, the OAuth2 authorization code flow must be followed for authentication.

Extract from a mail I received from Netatmo

[jostrasser]

Adding documentation from mail: https://dev.netatmo.com/apidocumentation/oauth#authorization-code

[skrollme]

Thanks for opening the issue. I also got the infomail from Netatmo. I will take a look at it soon...

[skrollme]

Since I still had no time to take a look at this refactoring and also no good starting point to integrate a web-redirect-based authentication in the plugin, it looks like it won't be done by Netatmo's deadline this October.

So be aware, that this plugin could be stop working and/or breaking your Homebridge setup any time soon.

If anyone is interested in participating here feel free to start coding and opening a PR in this repository.

[jostrasser]

Hi @skrollme that is really bad... right now there is no alternative to integrate Netatmo Weather into Homebridge. And you have done a really good job :)

I contacted Netatmo Dev Support to get more details about the new auth method, as I saw that users are now able to create an "access token" via the web portal in addition to the "client_id" and "client_token". This means a web-based redirect is maybe "not a must" for now. You only have to refresh the token periodically:

Our development team are going to allow the creation of the refresh and access token directly from the dev.netatmo.com account owning the application. This means that you will only need to manually generate it once, then you will simply be able to refresh it and save the new ones after refresh.

At the moment the token validity is 3 hours.

Will this makes it easier to keep the plugin "alive"? I hope this can help to solve the issue.

Thank you!

[jostrasser]

Hi @skrollme I want to add some more informations from Netatmo:

Note that all script that are actually using the refresh token method instead of fully remaking the client credential (that should be used once to create the token and its access level) will continue to work.

Eitherway for the node if they can save emails and password, saving the key is identical. We will however also need to change both the refresh and access key with each refresh in the near future. They will only need to copy paste the new access and refresh with every refresh. Here is how to refresh the token: https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

[skrollme]

Thanks for your investigation, @jostrasser I will take a look at the possibilities we have with your findings this weekend.

[jostrasser]

Thanks for your investigation, @jostrasser I will take a look at the possibilities we have with your findings this weekend.

Many thanks @skrollme !

[skrollme]

Just checked https://dev.netatmo.com/apps/ and to be honest I'm only able to generate access-token in my app's edit-form. No RefreshToken! And in this case it does not really help unless you want to manually generate a new access-token every 3 hours and put it into your homebridge setup :D

Or did you find out where you can generate access- AND refresh-token, @jostrasser ?

[juani13973]

Hi skzollme, thanks for your effort. I just created a token and it works again. I am not a technical person so I am not able to have a proper conversation with Netatmo but would it be worth asking if they can make the token last 3 months at least ?

[jostrasser]

@skrollme

The access token can be generated here: https://dev.netatmo.com/apps/ And I found the documentation about the token refresh process here: https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

I think this "should" be triggered within 3 hours periodically to stay authorized.

[jostrasser]

Hi skzollme, thanks for your effort. I just created a token and it works again. I am not a technical person so I am not able to have a proper conversation with Netatmo but would it be worth asking if they can make the token last 3 months at least ?

Netatmo confirmed that the Access token will be permanent but the refresh token has to be refreshed within every 3 hours.

Some statements:

Our developement team are going to allow the creation of the refresh and access token directly from the dev.netatmo.com account owning the application. This means that you will only need to manually generate it once, then you will simply be able to refresh it and save the new ones after refresh.

At the moment the token validity is 3 hours.

At the moment you do not need to recreate a new token, you can refresh it ad ifinitum once you have a valid token. This is however also prone to change in the future, also to improve the security of the devices. At the moment I can't confirm if the refresh token, the access token or both will have to be changed every three hours as we are still considering the best option.

What you can do is setting up a server on your device hosting the code and access it via your smartphone or any display you would have. Your mobile phone has to be on the same network than the device hosting your code.

The device should then redirect the call directly to https://api.netatmo/com/oauth2/authorize?client_id=''&redirect_uri=''&scope='' and setting the redirect uri as the locale ip of your device 192.168.xx. It does not need to be accessible from the outside but only on your local network.

On the mobile phone, you'd then be redirected on Netatmo front and it will ask you whether you want to gave access to the API app. When you'll click on 'Yes, I accept', Netatmo servers will send the code to Netatmo frontend which will redirect it to the locale ip from the redirect uri. In the end https://192.168.xx/?code='' will be received by the device hosting your development and will then be able to get the pair of tokens (access tokens& refresh token).

Note that if you already have a valid refresh and access token, you do not need to redo the access.

Feel free to let me know should you have any further request.

I hope this can help you :)

[jostrasser]

I just created a token and it works again.

Hi @juani13973

where did you refreshed the token and what exactly "works again"? At the moment Netatmo is not restricting the old auth method and will do this starting October.

This means all 3rd party apps / plugins access are not impacted right now but has to be adopted (Oauth2 authentication) to keep them working.

And homebridge-eveatmo is one of it.

[julien94800]

Hello @jostrasser From my side my log was full of the following message

I just logged into Netatmo dev portal and generated a token. Now it works again in Homebridge. Not sure if it's really this which fixed the situation.... but sounds strange

[jostrasser]

Hello @jostrasser From my side my log was full of the following message

I just logged into Netatmo dev portal and generated a token. Now it works again in Homebridge. Not sure if it's really this which fixed the situation.... but sounds strange

Ah, I am aware of this issue (run into it a few months ago) It's a platform issue on the Netatmo servers. Creating a fresh "secret" should solve the issue.

FYI: This isn't related to the auth method change which Netatmo is starting in October.

[juani13973]

Thx julien94800. Same with me. Same message in the console and home bridge had lost access to fetch the data. The temp, humidity etc did not work. I went into the Netatmo dev website where I had created the connection to the plug in and there is a link to create a token. I just clicked, the token was created and "magic" the plugin started working again.

I guess based on the conversation here that I need to do this token refresh every 3 hours with is impossible. once every 3 month would be OK

[jostrasser]

Hmm, the "access token" (Oauth2) should not be related to your issues because when you generate a token you have to enter it in your json.conf of Homebridge, otherwise it will not be used... and this is not possible right now ;)

But creating a token can force a sync of the authentication change @ Netatmo´s Platform. And this can solve such issues.

THIS token will be needed starting October To use it we have to adopt some changes in homebridge-eveatmo plugin:

[julien94800]

Hello @jostrasser Yes that's exactly on this button I clicked. Then I get tokens (I assume the access one and the refresh one separated by |) Indeed I did not used them anywhere.... but by any reason it unlocked my trouble. Surely something strange on Netatmo server.

Thanks anyway, and let's hope we could find a way to get this plugin still working on future!

[jostrasser]

Hello @jostrasser Yes that's exactly on this button I clicked. Then I get tokens (I assume the access one and the refresh one separated by |) Indeed I did not used them anywhere.... but by any reason it unlocked my trouble. Surely something strange on Netatmo server.

Thanks anyway, and let's hope we could find a way to get this plugin still working on future!

Hi @julien94800 Thanks for your confirmation :)

Yes, I think Netatmo is having troubles right now on their side and the described steps are solving it. However, I also hope we can find a way to integrate the new authentication method.

[skrollme]

Normally you get access- and refresh-token after auth. You can use the access-token as long as it is valid. If it becomes invalid you need to use the refresh-token to get a new pair of token and everything starts over again.

But when I generate a token on https://dev.netatmo.com/apps/ I only get an access-token, no refresh-token. Not sure if Netatmo does something special here or am I missing anything, but to only have the access-token seems useless for long-term usage.

[jostrasser]

Normally you get access- and refresh-token after auth. You can use the access-token as long as it is valid. If it becomes invalid you need to use the refresh-token to get a new pair of token and everything starts over again.

But when I generate a token on https://dev.netatmo.com/apps/ I only get an access-token, no refresh-token. Not sure if Netatmo does something special here or am I missing anything, but to only have the access-token seems useless for long-term usage.

Hi @skrollme

I have an open ticket at Netatmo's Dev Support. I can request some more informations about it.

[skrollme]

But from your statements it looks like we have the current situation now: 1) refresh-token generation does not work at the moment, but it is coming? 2) access-token (which normally expire) do not expire

If 2) is the case, it could be an easy (temporary) fix to keep the plugin running. I will generate a token now and check if it still works tomorrow.

[skrollme]

@juani13973 and @julien94800 The problems you are describing does not seem to be related to this topic/issue here. Netatmo seems to have some general and unrelated problems with their infrastructure which happens every now and then.

[jostrasser]

But from your statements it looks like we have the current situation now:

1. refresh-token generation does not work at the moment, but it is coming?
2. access-token (which normally expire) do not expire

If 2) is the case, it could be an easy (temporary) fix to keep the plugin running. I will generate a token now and check if it still works tomorrow.

Hi @skrollme I have requested a confirmation from Netatmo for your questions.

1. waiting for confirmation from Netatmo
2. I think if a access token is created (via dev.netatmo.com) only the refresh token has to be "refreshed" periodically by homebridge-eveatmo in background. Netatmo will change this in the future (not official confirmed now) but for now this procedure should work. If they are changing also this procedure they will inform the customers again (hope so) ;)

[skrollme]

The access-token did not survive the night, so the "easy solution" won't work. I will start evaluating other ways...

[jostrasser]

Feedback from Netatmo Dev Support:

1) refresh-token generation does not work at the moment, but it is coming? 2) access-token (which normally expire) do not expire

---

1) The token generation works, it's the refresh token generation directly from dev.netatmo.com application's webpage that is coming. This is to allow people to generate the token if the machine running the script is a pi for exemple. They will only need to copy/paste the key then refresh the token & save the new token to replace the old one.

2) The access token do expire after 3 hours, this is why the refresh is recommended instead of doing the aouth/cred process all over again.

[skrollme]

I just releases a new beta-version (0.7.0-beta2) of this plugin which uses a _refreshtoken for authentication.

• The new version only needs a _clientid, a _clientsecret (like before) and a valid _refreshtoken for this client.
• Upon startup the plugin uses the _refreshtoken to get an _accesstoken which is then used in all API-calls like getStationData
• The plugin also tries to fetch a new _accesstoken when it is about to expire (every 3h at the moment)

A first and short test went fine so I'm about to install it on my raspberryPi to use it at home because it definitely needs some long-time testing. Everyone who is able to generate a _refreshtoken (also not officially supported in the https://dev.netatmo.com/apps/ console) is also invited to do this and test the beta-version.

[julien94800]

Hello @skrollme Thanks very much for your time on this subject. I would be glad helping testing too but I have no idea on how to generate a refresh token at the moment. Any hint on this please? Thanks in advance

[skrollme]

Could be a little bit tricky without knowledge of the authentication-mechanism and no experience in simulating these auth-flows via an API-testing-tool like postman, but I appreciate your willingness, @julien94800.

@jostrasser Did Netatmo's support say anything about if or when _refreshtoken expire as well? At the moment the plugin does not update the _refreshtoken when it fetches a new _accesstoken, but it could be necessary if they also expire.

[julien94800]

Ok thanks for your feedback so I will wait. I was just hoping being able to get it from Netatmo website. When I generate tokens on their site I always get two tokens (they are separated by | character). Do you know what are they? I assume first one is the access token but no idea about the second one. Could it be a refresh one?

[jostrasser]

Hi @skrollme Many thanks for your great work to keep the plugin alive.

About the tokens, I got the following information from Netatmo Dev Support:

I understand that a few months is a bit shorter than usual, I naturally already transmited this to the team in charge. It is however unlikely that his delay will be pushed back at the moment. Note that all script that are actually using the refresh token method instead of fully remaking the client credential (that should be used once to create the token and its access level) will continue to work.

Eitherway for the nodes / plugins if they can save emails and password, saving the key is identical. We will however also need to change both the refresh and access key with each refresh in the near future. They will only need to copy paste the new access and refresh with every refresh. Here is how to refresh the token: https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

I'm on vacation until next week and will start testing immediately when I am back home. :)

[skrollme]

We will however also need to change both the refresh and access key with each refresh in the near future.

Ok so I have to take a look at this as well :D Thanks for forwarding this, @jostrasser

[jostrasser]

We will however also need to change both the refresh and access key with each refresh in the near future.

Ok so I have to take a look at this as well :D Thanks for forwarding this, @jostrasser

No problem @skrollme !

If additional infos are required let me know ;)

[skrollme]

Since the beta2 worked for a few days straight without any problems I published a next iteration (0.7.0-beta3) which also stores the eventually updated _refreshtoken after every _accesstoken-renewal. This is done via persisting it into a config-file, next to the homebridge-config. This ensures, that homebridge is still able to query the Netatmo-api even after the _refreshtoken from the homebridge-config is expired.

For more info please see the Readme.md of the test branch and try the beta3 on your own if you have the possibility to generate a _refreshtoken on your own.

[palasinio]

Hi @skrollme, are you able to describe the way to get the "refresh" token more exactly? I tried the token generated on the /dev web page with, w/o, left from and right from the "|". Nothing worked. Some screens maybe helpful.

Big THX for still going on

[skrollme]

Since the auth-flow needs some requests/redirects and form-submits it is not possible without some tools. I will try to build a little test-page where you can enter your app's client-id to initiate the auth-flow.

If you are still interested in this topic and want to dig deeper own your own @palasinio, you can checkout out this video, which does a basic explanation of the auth-mechanism and introduced the tool Postman, which can be used to do the auth-flow. But you have to adapt the process to Netatmo's API servers: https://www.youtube.com/watch?v=NRU_KdUSjD4

[jostrasser]

Hi @skrollme What is the goal after finalizing this version with integrated refresh of both (refresh and access) tokens? Must users of the plugin initiate the auth process manually as described (e.g. via Postman) or are you trying to integrate it directly into the plugin so that users only have to enter the available IDs from dev.netatmo.com?

[jostrasser]

Hi @skrollme I have now updated to 0.7.0-beta3 and have generated the needed tokens with postman. The plugin is running fine for now.

I'll keep you posted about the testings.

If anyone needs help to generate the token with postman: I can post a screenshot of the configuration if needed.

Thanks JO!

[palasinio]

@jostrasser: what is „postman“? A screen would be very helpful :)

[jostrasser]

@palasinio Postman is a API Platform Client... download it from here: https://www.postman.com

Then you can do the following, please fill in your information on the fields "Token Name, Client ID and Client Secret":

You have to set the following on the "Scope" field to get full read access to all Netatmo resources (copy/paste it):

read_station read_magellan read_bubendorff read_smarther read_thermostat read_camera read_doorbell read_mx read_presence read_homecoach read_carbonmonoxidedetector read_smokedetector

Then click "Get New Access Token".

After a successful authentication via your webbrowser you will be redirected back to Postman and you will get the needed tokens. Then you can copy them into your Homebridge config :)

[skrollme]

What is the goal after finalizing this version with integrated refresh of both (refresh and access) tokens? Must users of the plugin initiate the auth process manually as described (e.g. via Postman) or are you trying to integrate it directly into the plugin so that users only have to enter the available IDs from dev.netatmo.com?

Since I do no not how an initial authentication could be done within the plugin easily, the initial setup will be similar to the current state. So a user has to generate a _refreshtoken and use it on the first start. Hopefully the dev.netatmo.com portal will support this soon. Otherwise I have to prepare a small page which can initiate the authflow and prints out a _refreshtoken which can be used

[jostrasser]

What is the goal after finalizing this version with integrated refresh of both (refresh and access) tokens? Must users of the plugin initiate the auth process manually as described (e.g. via Postman) or are you trying to integrate it directly into the plugin so that users only have to enter the available IDs from dev.netatmo.com?

Since I do no not how an initial authentication could be done within the plugin easily, the initial setup will be similar to the current state. So a user has to generate a _refreshtoken and use it on the first start. Hopefully the dev.netatmo.com portal will support this soon. Otherwise I have to prepare a small page which can initiate the authflow and prints out a _refreshtoken which can be used

Okay, thanks for this information.

I will ask the Netatmo Dev Support for it again / open a Feature Request.

[palasinio]

@jostrasser Big THX for your instructions. Important to say: It is not nescessary to register to "PostMan"(!). The plugin at least starts w/o error or warning :)

@skrollme like described in https://dev.netatmo.com/apidocumentation/oauth I just tried to play /w POST and pasting the URLs in my browser w/o knowing what I'm doing, but I bet there is a way to get the refresh token whith some posts.

Duno if plugin can do this, cause netatmo is requesting ack. As there are no dataupdates written to the log anymore I gues, I can tell tomorrow, if it works ;)

[jostrasser]

@jostrasser Big THX for your instructions. Important to say: It is not nescessary to register to "PostMan"(!). The plugin at least starts w/o error or warning :)

@skrollme like described in https://dev.netatmo.com/apidocumentation/oauth I just tried to play /w POST and pasting the URLs in my browser w/o knowing what I'm doing, but I bet there is a way to get the refresh token whith some posts.

Duno if plugin can do this, cause netatmo is requesting ack. As there are no dataupdates written to the log anymore I gues, I can tell tomorrow, if it works ;)

Hi @palasinio No problem. Postman works without creating an account, you only need it if you want to sync your work over multiple instances.

It is enough to request the keys via Postman (do an authentication to get the keys back as described) which you can use in your homebridge config file. You only have to set:

"auth": {
                "client_id": "xxx",
                "client_secret": "xxx",
                "refresh_token": "xxx"

After that the plugin will create the netatmo-token.jsonfile to store the keys in the homebridge directory (typically /var/lib/homebridge).

@skrollme I am running on beta3 since two days without any issues. :) If I verify the netatmo-token.json file I can see that it is changing the key every day correctly.

[andoorius]

Hmm, the "access token" (Oauth2) should not be related to your issues because when you generate a token you have to enter it in your json.conf of Homebridge, otherwise it will not be used... and this is not possible right now ;)

But creating a token can force a sync of the authentication change @ Netatmo´s Platform. And this can solve such issues.

THIS token will be needed starting October To use it we have to adopt some changes in homebridge-eveatmo plugin:

I‘m not able to use the Plugin Beta 3… Token were generated in the Apps section of the Netatmo Webpage. Client ID,Secret and Token generated new several times. Anyone else facing this issue?

[1.10.2022, 12:54:38] [eveatmo platform] Starting child bridge... [1.10.2022, 12:54:40] [eveatmo platform] Launched child bridge with PID 29760 [1.10.2022, 12:54:40] Registering platfor 'homebridge-eveatmo.eveatmo 1 [1.10.2022, 12:54:40] [eveatmo platform] Loaded homebridge-eveatmo v0.7.0-beta3 ch ild bridge successfully [1.10.2022, 12:54:40] Loaded 0 cached acc essories from cachedAccessories.0E6F9CC49 5C7. [1.10.2022, 12:54:43] [eveatmo platform] Loading new data from API for: weathersta tion [1.10.2022, 12:54:43] [eveatmo platform] Loading new data from API for: airquality [1.10.2022, 12:54:43] [eveatmo platform] WARN - Netatmo: Error: Authenticate refre sh error: invalid grant 11.10.2022, 12:55:001 Thomebridge-eveatmo ] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR

[jostrasser]

@andoorius You are facing an auth-issue. With beta3 you have to verify that both, the access AND the refresh token, are configured correctly to initiate the connection.

I ran into a similar issue and found out that one token wasn't correctly applied to the netatmo-token.json because I created them twice but the netatmo-token.json will not be overwritten.

So I will recommend the following:

1) check your HB config and verify the correctness of the client_id, client_secret and refresh_token. 2) check the netatmo-token.json in the working dir and verify that BOTH tokens are set up correctly.

If not: request a fresh one with Postman (Postman will deliver you both tokens (access and refresh) in one screen after a successful auth via your browser) and copy them over again.

Restart your HB again.

Now, after running on beta3 for three days and another "restart test" today I can confirm that the beta3 is working and refreshing the access token correctly (daily) on my Homebridge.

Hope this helps.

[andoorius]

Well somehow 😀 It is probably a stupid question Where should I find the Netatmo-token.json? Also search did not help me :-(

[jostrasser]

Well somehow 😀 It is probably a stupid question

Where should I find the Netatmo-token.json?

Also search did not help me :-(

Please read my earlier comments...

---

After that the plugin will create the netatmo-token.json file to store the keys in the homebridge directory (typically /var/lib/homebridge).

[andoorius]

ok I had issues with the Token creation. Now it works. The Netatmo-token.json is only created after one successful authentication... Thank you!

[jostrasser]

ok I had issues with the Token creation. Now it works. The Netatmo-token.json is only created after one successful authentication...

Thank you!

Yes, that's correct.

No problem, U're welcome :)