Authentication failed

[WayneManion]

I have been using this Sensi integration for a few weeks. Today I had a notification that the authorization for the Sensi integration was expired. I went to the Integrations page in Home Assistant and clicked "Re-configure" for the Sensi integeration.

A modal box with no text and just a "sleepy eyeball" appeared. I entered my Sensi account password. Authentication failed. I went to the mobile app on my phone and changed the password. I was able to log into the mobile app with the new password. I used the new password in the Sensi integration and again, authentication failed.

[715Steve]

This started for me as well. Removed and reinstalled but cannot get past the credentials screen. Authentication failed.

[iprak]

Most probably Sensi backend changed the verification, this has happened before https://github.com/iprak/sensi/issues/28.

[bperry11]

+1 for having this issue pop up today.

[NathanT15]

Same here, I got the auth request, tried my password, and then removed and reinstalled but cannot get past the credentials screen.

[derousse]

Same for me...

[sjbarkey]

Issue started today for me as well.

[KE2EAC]

Same here.....

[Menz01]

Same issue for me too

[laurenslo]

same.

[stleusc]

There is a new header required:

rct: xxxxxxx

It changes with every login based on MITMPROXY and Android app. The code to create the value seems to be at least partially in com.emerson.sensi.util.recaptcha

So far I don't know yet how exactly it is calculated....

[stleusc]

I had filtered to traffic with Sensi only, haha. There are calls to:

https://www.recaptcha.net/recaptcha/api3/mri
https://www.recaptcha.net/recaptcha/api3/mrr
https://www.recaptcha.net/recaptcha/api3/mlg

It seems like the first one gets static data (mostly) and the response to mrr contains the header we need for rct. Need some more digging but should be fairly straight forward.

Maybe someone finds docu for that API?

SEEMS to be Google Reccaptcha V3 (maybe)

[jsiemonski]

Add me to the pile 😁

[squid1127]

I am having this issue as well

[mondomondoman]

I am also having this issue.

[stephennutt]

+1

[jcamm3]

+1 same issue

[stleusc]

I really don't think any more +1 comments are really helping the issue. This is going to happen for EVERYONE! I am pretty sure the repo owner got that point and maybe we should limit comments to things that are helpful to fixing the issue.

[iprak]

The way I understand reCAPTCHA is the app talks to captcha server using an app specific key. The server returns a token based on bot identification and tells the backend server. This token is then passed to the backend server during authentication. I am suspecting the rtc header is this token.

I have been looking at the decompiled app and have not been able to find any key so far. I will keep digging.

https://cloud.google.com/recaptcha-enterprise/docs/instrument-android-apps https://www.geeksforgeeks.org/how-to-integrate-google-recaptcha-in-android/

[stleusc]

Key needed:

6LdS6SYpAAAAAFCti9uo4Yg47fSIGygzTGl_6lEV

This is from Android app...

[stleusc]

I can create a fake POST to mri endpoint. It contains a few values needed for mrr endpoint. mrr takes a total of 7 strings protobuf encoded. String 1 and 2 are in response from mri, String 3 is the key and String 4 is "login" String 5-7, so far no clue what they are or where to get them.... Once we have those we can post to mrr and the response will contain the value needed for the auth header that we are missing now.

[troyboy27]

+1 - but don't understand how to fix it.

[calimansi]

The integration through homekit still works, but doesn't have as many options.

[omarquis]

+1 how can I fix the issue?

[iprak]

@troyboy27 @omarquis This cannot be addressed without code changes. The authentication at Sensi end has changed.

[macwriter]

I am also investigating the {"grant_type":"refresh_token","refresh_token": "some_random_string"} , if the refresh_token is long lived and how to acquire it. More than likely it will rely on the aforementioned recaptcha.

Will report when I know more...

data = {
        "client_id": client_id,
        "client_secret": client_secret,
        "grant_type": "refresh_token",
        "refresh_token": refresh_token
    }

headers = {
        "accept": "*/*",
        "accept-language": "en-US,en;q=0.9",
        "content-type": "application/x-www-form-urlencoded; charset=utf-8"
    }

[stleusc]

More than likely it will rely on the aforementioned recaptcha.

You get that as part of the auth process. You generally get the token and a refresh token. Once the token expires you exchange the refresh token for a new pair. That step requires the recaptcha as well.

BUT, Google has libraries that take care of that on Android, not sure if Sensi had to implement sth for iOS or if there is a library or even a different process... Since it does not matter for us we might get lucky with iOS and the way it's done there. Can you log traffice between the app and sensi/google? Using sth like MITM proxy or so.

That is how I found out about the recaptcha process...

[macwriter]

Ok thanks,

I got the "refresh_token" by logging the traffic via MITM proxy and have been using that in the sensi service for HA as a workaround.

I did see the recaptcha as well., but I could not follow the output. I will have another try at it and report back later. Thanks for the tips!

[stleusc]

I'll take some of what I said back... once you have the refresh tokoen you do not need recaptcha anymore!

[stleusc]

have been using that in the sensi service for HA as a workaround.

I tried that and it is failing... how did you tell HA to use it? I tried to enter it into the box that asks for auth.... but not working.

[macwriter]

looks like it does the following

https://www.recaptcha.net/recaptcha/api3/mri https://firebaselogging-pa.googleapis.com/v1/firelog/legacy/batchlog https://sensiapi.io/status https://oauth.sensiapi.io/token?device=XXXX

[macwriter]

have been using that in the sensi service for HA as a workaround.

I tried that and it is failing... how did you tell HA to use it? I tried to enter it into the box that asks for auth.... but not working.

I tested the methodology outside HA and then provided it via configurable variable inside the auth.py. Maybe @iprak can expose this variable in the configuration.

post_data = {
        # "username": config.username,
        # "password": config.password,
        "client_id": client_id,
        "client_secret": client_secret,
        "refresh_token": refresh_token,
        "grant_type": "refresh_token",
    }
headers = {
            "accept": "*/*",
            "accept-language": "en-US,en;q=0.9",
            "content-type": "application/x-www-form-urlencoded; charset=utf-8",
        }

[stleusc]

That makes sense....

---

From: Joe King @.> Sent: Monday, March 11, 2024 1:43:06 PM To: iprak/sensi @.> Cc: Stephan Leuschner @.>; Comment @.> Subject: Re: [iprak/sensi] Authentication failed (Issue #38)

have been using that in the sensi service for HA as a workaround.

I tried that and it is failing... how did you tell HA to use it? I tried to enter it into the box that asks for auth.... but not working.

I tested the methodology outside HA and then provided it via configurable variable inside the auth.py. Maybe @iprakhttps://github.com/iprak can expose this variable in the configuration.

— Reply to this email directly, view it on GitHubhttps://github.com/iprak/sensi/issues/38#issuecomment-1989057015, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIUMSUVBUOK4K5WL77ON6LYXX3LVAVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBZGA2TOMBRGU. You are receiving this because you commented.Message ID: @.***>

[troyboy27]

WOW! Thanks for all the research Stephan and Joe! I'm more then willing to test any code y'all come up with on my system.

On Mon, Mar 11, 2024 at 12:46 PM Stephan Leuschner @.***> wrote:

That makes sense....

---

From: Joe King @.> Sent: Monday, March 11, 2024 1:43:06 PM To: iprak/sensi @.> Cc: Stephan Leuschner @.>; Comment @.> Subject: Re: [iprak/sensi] Authentication failed (Issue #38)

have been using that in the sensi service for HA as a workaround.

I tried that and it is failing... how did you tell HA to use it? I tried to enter it into the box that asks for auth.... but not working.

I tested the methodology outside HA and then provided it via configurable variable inside the auth.py. Maybe @iprakhttps://github.com/iprak can expose this variable in the configuration.

— Reply to this email directly, view it on GitHub< https://github.com/iprak/sensi/issues/38#issuecomment-1989057015>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AAIUMSUVBUOK4K5WL77ON6LYXX3LVAVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBZGA2TOMBRGU>.

You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/iprak/sensi/issues/38#issuecomment-1989063594, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHE2LI4VXFN7L7G5HDKDZLYXXUVXAVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBZGA3DGNJZGQ . You are receiving this because you were mentioned.Message ID: @.***>

[iprak]

@macwriter

Sensi has removed previous authentication mechanism. Even the previous Android app 8.6.2 fails to login. My initial hope was that there was a fall back mechanism tied to the appVersion passed as header. Even faking "ios" did not help.

I am not seeing how would one get the refresh_token ?

[stleusc]

All my work is based on the latest android version 8.6.3 You get the refresh_token as part of the auth flow. 'All' we need is the recaptcha value...

[iprak]

Oh I misread that comment .. I thought that recaptcha value was not needed.

The Android client uses a WebView to host reCaptcha content. I will play with some Python libraries which offer reCaptcha implementation.

[macwriter]

@macwriter

Sensi has removed previous authentication mechanism. Even the previous Android app 8.6.2 fails to login. My initial hope was that there was a fall back mechanism tied to the appVersion passed as header. Even faking "ios" did not help.

I am not seeing how would one get the refresh_token ?

It does not appear that the 'grant_type':'refresh_token' is working for me anymore.

The app is now using the {'grant_type': 'password'} with the rpt: header.

[Pheelix]

Not sure if this helps, but I went to there website and signed in with inspect open. Did a search for token on the network and saw this for a device path listing. The ***** after FL33T is my accounts username.

Headers Tab Request URL: https://oauth.sensiapi.io/token?device=FL33T-*****-581025838

:path: /token?device=FL33T-*****-581025838

Payload Tab client_id: fleet

[stleusc]

Which website?

---

From: Pheelix @.> Sent: Tuesday, March 12, 2024 5:44:58 AM To: iprak/sensi @.> Cc: Stephan Leuschner @.>; Comment @.> Subject: Re: [iprak/sensi] Authentication failed (Issue #38)

Not sure if this helps, but I went to there website and signed in with inspect open. Did a search for token on the network and saw this for a device path listing. The ***** after FL33T is my accounts username.

Headers Tab Request URL: https://oauth.sensiapi.io/token?device=FL33T-*****-581025838

:path: /token?device=FL33T-*****-581025838

Payload Tab client_id: fleet

— Reply to this email directly, view it on GitHubhttps://github.com/iprak/sensi/issues/38#issuecomment-1991193041, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIUMSQTAWHQFJV4U2EDK63YX3MCVAVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJRGE4TGMBUGE. You are receiving this because you commented.Message ID: @.***>

[Pheelix]

https://manager.sensicomfort.com/

It's the same site someone would use if they paid for there subscription to access there thermostats from there site.

username/password is the same on both app and website.

[stleusc]

That's what I thought... Are you paying? I assume there is no functionality if you don't pay, right?

---

From: Pheelix @.> Sent: Tuesday, March 12, 2024 7:45:17 AM To: iprak/sensi @.> Cc: Stephan Leuschner @.>; Comment @.> Subject: Re: [iprak/sensi] Authentication failed (Issue #38)

https://manager.sensicomfort.com/

It's the same site someone would use if they paid for there subscription to access there thermostats from there site.

— Reply to this email directly, view it on GitHubhttps://github.com/iprak/sensi/issues/38#issuecomment-1991464508, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIUMSQE6UXRTQB3EOT4OYLYX3TEZAVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJRGQ3DINJQHA. You are receiving this because you commented.Message ID: @.***>

[Pheelix]

nope, not paying, so no functionality. But i figured it was worth a shot to see if it would help at all. Tho atm I am using the Homekit method in HA to control my Sensi Touch. But would be nice to see this up and running again.

[stleusc]

It's definitely a good idea... Guess someone needs to pay to see what the actual API calls are and then test it with someone that's not paying to see if it works.

---

From: Pheelix @.> Sent: Tuesday, March 12, 2024 7:48:45 AM To: iprak/sensi @.> Cc: Stephan Leuschner @.>; Comment @.> Subject: Re: [iprak/sensi] Authentication failed (Issue #38)

nope, not paying, so no functionality. But i figured it was worth a shot to see if it would help at all. Tho atm I am using the Homekit method in HA to control my Sensi Touch. But would be nice to see this up and running again.

— Reply to this email directly, view it on GitHubhttps://github.com/iprak/sensi/issues/38#issuecomment-1991469817, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIUMSS5VJO4XZSUOQCIARTYX3TR3AVCNFSM6AAAAABEHWELPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJRGQ3DSOBRG4. You are receiving this because you commented.Message ID: @.***>

[Pheelix]

site says first month is free, cancel any time. so I guess it could be tested without having to pay for it. Just remember to cancel before the first month ends.

[stleusc]

Actually, I don't think that is even needed. Just looked at the traffic.... The response looks VERY similar to the app and contains the required tokens.

@macwriter you already have modified sources... I think it would be super easy for you to check if the response tokens work. Simply log into https://manager.sensicomfort.com/ and look at the traffic to grab the token from the oauth call. I have a good feeling about this!

@Pheelix you might have solved the mystery for us...

@iprak The website also uses recaptcha but it is fully browser based. I ASSUME that python can emulate a webbrowser in a way that you load the login page and let it do it's thing to take care of the recaptcha?

[stleusc]

I can confirm, the tokens work!

Changed auth.py around line 67

    refresh_token = "copied from browser response"
    client_id = "fleet" # instead of android
    client_secret = "JLFjJmketRhj>M9uoDhusYKyi?zUyNqhGB)H2XiwLEF#KcGKrRD2JZsDQ7ufNven" # different secret

    post_data = {
        #"username": config.username,
        #"password": config.password,
        #"client_id": CLIENT_ID,
        #"client_secret": CLIENT_SECRET,
        #"grant_type": "password",
        "client_id": client_id,
        "client_secret": client_secret,
        "refresh_token": refresh_token,
        "grant_type": "refresh_token",
    }

Error in HA goes away after a restart...

I guess if @iprak can simulate the browser login then we are back in action!

I guess worst case solution would be to have the user login to the website and copy the refresh token and have that as an external config value that we have to paste. I guess other add-ons do the same where the user needs to create an account with company X and provide some keys manually.

I guess my manual fix will stop working as soon as my current token expires since I hard-coded the refresh token and it only works once. But if we can feed the initial refresh_token and than the addon keeps managing it like in the past it should work long term.

@Pheelix amazing catch!

[stleusc]

@iprak I just looked at the code and noticed that it seems like you never really use the refresh_token. Is that right? If I understood it right you are sending user/pwd again when the original token expires using "grant_type": "password". That is not the intention of OAuth... Ideally your addon would never know my user/pwd and only save the 2 tokens. Once the access token expires you ask for a fresh one using "grant_type": "refresh_token" and the saved refresh_token value. That will give you a new pair and you repeat. You only ever need user/pwd ONCE at the very beginning.... Then never again. With this changed flow the user could provide a single valid refresh_token and no user/pwd and you don't have to worry about recaptcha since that seems to be needed only for the user/pwd exchange!

[iprak]

Python cannot emulate reCaptcha, some library attempt to do that but they either implement v2 or are paid. reCaptcha involves a rotating sort of token which is generated by reCaptcha library.

Sensi uses the latest google reCaptcha. In web, this e.g. https://www.google.com/recaptcha/enterprise.js?render=6LebYk0pAAAAAIUlMrqCdXOq83pY2O4u-CxuTh28 returns an executing block which returns the token.

The ideal way to address would be to change the initial authentication itself. This is how authentication is implemented for 3rd party verification e.g. https://sensiapi.io/authorize

In my experimentation, extracting token from web and passing to Python did not work. There might be additional headers/cookies.

Yes, you are right about refresh_token. I just login in again; I could not confirm when refreshing would be adequate.

[stleusc]

You refresh when the page rejects the auth token. So instead of login again, you refresh. Can Python emulate a webbrowser? If so, it would be easy to load the login page, fill in user/pwd and grab the tokens.

Not sure why you said using the extracted token in python does not work? See my prior comment.. it DOES work 100%! But you need more than the token! There are 2 other values that change. Both are in my code snippet (client_id and client_secret).

[stleusc]

I see two possible solutions here:

A: Change the setup flow to ask the user for a refresh_token. Explain that the user can get it by logging into https://manager.sensicomfort.com/ and then copying the value using the browser developer tools. This step is needed ONCE to setup. After that the refresh_token can be exchanged for an auth_token and a new refresh_token. Whenever sensi complains about expired credentials simply exchange again. This is the simplest impementation and would not require much change I think?!

B: Have python emulate a webwroser and login to https://manager.sensicomfort.com/ using username/pwd and then grab the refresh_token from the web traffic. Would be easy in and Android WebView, not sure if Python can do that. Once you have that token the flow should be like in A. Don't store user/pwd but only refresh_token and renew when needed.

This hardcoded snippet DOES work until the token expires (hard coded refresh_token works only once. But by replacing the hard-coded token with one initially provided by the user and then updated on every refresh this really works. I see all my thermostats in HA as we speak!

auth.py around line 68ish

refresh_token = "copied from browser response"
client_id = "fleet" # instead of android
client_secret = "JLFjJmketRhj>M9uoDhusYKyi?zUyNqhGB)H2XiwLEF#KcGKrRD2JZsDQ7ufNven" # different secret

post_data = {
    #"username": config.username,
    #"password": config.password,
    #"client_id": CLIENT_ID,
    #"client_secret": CLIENT_SECRET,
    #"grant_type": "password",
    "client_id": client_id,
    "client_secret": client_secret,
    "refresh_token": refresh_token,
    "grant_type": "refresh_token",
}