Closed craibo closed 1 month ago
lostfields
commented
3 months ago I've seen that Panasonic are trying to migrate user over to use OAuth2 where you can link your account to Google etc. If this is going forward, I think it will be harder and harder for us to have a library like this without support for third-party developers by Panasonic.
majurgens
commented
3 months ago It looks like the HA community have identified what has changed but would rely on coding done in this project to resolve. https://github.com/sockless-coding/panasonic_cc/issues/191#issuecomment-2174505567
nelsongraca
commented
3 months ago @majurgens I'll be looking a little into the HA Integration the trickiest part might be a user friendly way of getting the oauth code, I can do it through postman or curl, an option would be doing the full login process programmatically this way we can control the flow and ignore redirects, we would only need it once though after that it's standard oauth refresh.
heldchen
commented
3 months ago the accsmart api endpoint was updated to only support Bearer token it seems. the legacy token that one could fetch with the login & password is defunct.
by intercepting the traffic from the android app (for a recipie see https://github.com/sockless-coding/panasonic_cc/issues/191#issuecomment-2174505567), I found the following minimal workflows that need to be supported to get a bearer token and use it in subsequent calls. the workflows assume that you already fully configured your account in the app (f.e. enabled 2FA etc).
the app itself uses the auth0 SDK to do handle the oauth authorization. the authenticator webservice also publishes the standard oauth definitions (https://authglb.digital.panasonic.com/.well-known/openid-configuration), so maybe using a third party oauth client would be the easiest route...
POST to https://authglb.digital.panasonic.com/usernamepassword/login
Notes:
Request Header:
{"name":"auth0.js-ulp","version":"9.23.2"}Request Cookies:
Request Body:
{
"client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
"redirect_uri": "panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback?lang=en",
"tenant": "pdpauthglb-a1",
"response_type": "code",
"scope": "openid offline_access comfortcloud.control a2w.control",
"audience": "https://digital.panasonic.com/Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx/api/v1/",
"_csrf": "cIf...WjI",
"state": "hKF...KeA",
"_intstate": "deprecated",
"nonce": "dW3...OSg",
"username": "<my-email>",
"password": "<my-password>",
"lang": "en",
"connection": "PanasonicID-Authentication"
}Response Body: html form with 3 hidden inputs:
POST https://authglb.digital.panasonic.com/login/callback
Request Header:
Request Cookies:
Request Body: <url-encoded values of the form fields "wa", "wresult", "wctx" of the previous step>
Response Header:
Response Cookies:
GET https://authglb.digital.panasonic.com/authorize/resume?state=Bm6...T2O-
Request Cookies:
Response Header:
Response Cookies:
POST https://authglb.digital.panasonic.com/oauth/token
Request Header:
Request Body:
{
"scope": "openid",
"client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
"grant_type": "authorization_code",
"code": "vUc...o7n",
"redirect_uri": "panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback",
"code_verifier": "IEg...Ug4"
}Response Header:
Response Cookies:
Response Body:
{
"access_token": "eyJ...PwQ",
"refresh_token": "v1.M...yII",
"id_token": "eyJ...fHQ",
"scope": "openid comfortcloud.control a2w.control offline_access",
"expires_in": 86400,
"token_type": "Bearer"
}POST https://authglb.digital.panasonic.com/oauth/token
Request Header:
Request Body:
{
"scope": "openid offline_access comfortcloud.control a2w.control",
"client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx",
"refresh_token": "v1.M...OVg",
"grant_type": "refresh_token"
}Response Header:
Response Cookies:
Response Body:
{
"access_token": "eyJ...TSw",
"refresh_token": "v1.M...HYk",
"id_token": "eyJ...8XA",
"scope": "openid comfortcloud.control a2w.control offline_access",
"expires_in": 86400,
"token_type": "Bearer"
}POST https://accsmart.panasonic.com/auth/v2/login
Notes:
Request Header:
Request Body:
{
"language": 0
}Response Header
Response Body:
{
"country": "CH",
"extUsrId": "d04...b41",
"clientId": "048...a68",
"language": 0
}GET https://accsmart.panasonic.com/device/group
Request Header:
Response Header:
Response Body:
{
"uiFlg": false,
"groupCount": 1,
"groupList": [
...
]
}
danielcherubini
commented
3 months ago not yet clear where the Auth0-Client value comes from. could be a device-calculated unique identifier
@heldchen it might be this https://community.auth0.com/t/auth0client-parameter-in-the-authorize-request/114446
The auth0client parameter contains the telemetry information sent by the Auth0 SDK. It can be decoded with base64 to reveal the information it carries.
danielcherubini
commented
3 months ago so maybe using a third party oauth client would be the easiest route...
https://github.com/auth0/auth0-python
That might be the best route, if the above missing value is indeed comes from the Auth0 SDK, then using the above pythonSDK should provide it..
nelsongraca
commented
3 months ago so maybe using a third party oauth client would be the easiest route...
https://github.com/auth0/auth0-python
That might be the best route, if the above missing value is indeed comes from the Auth0 SDK, then using the above pythonSDK should provide it..
as someone who had to integrate with oauth, yes don't reinvent the wheel if a library exists use it.
mkz212
commented
3 months ago @heldchen Wow! Great work!
can someone write here this value?
In last step (Get ACCSmart Device Groups) X-User-Authorization-V2 it is token and it have login creditentials included?
In one step before last (Get ACCSmart Client-Id) you get client id but you dont use it in last step?
danielcherubini
commented
3 months ago @mkz212 thanks for finding a value of that header. Looks like I was right with what I found... It is just a base64 encoded of what SDK and version was used.
Decoded it is.
{"name":"auth0.js-ulp","version":"9.23.2"}
heldchen
commented
3 months ago I think that Auth0-Client is also hardcoded. Look here: https://github.com/cjaliaga/aioaquarea/blob/523aaba080e2f2e9e315de86ed728bcc270c6527/aioaquarea/core.py#L274C15-L274C32
great, added to the writeup
- In last step (Get ACCSmart Device Groups) X-User-Authorization-V2 it is token and it have login creditentials included?
it's a bearer token, a temporary token assigned to your user account that is valid for 24h, then needs to be refreshed. your credentials are not encoded in this, it's a separate token.
- In one step before last (Get ACCSmart Client-Id) you get client id but you dont use it in last step?
good catch, forgot to add it to the header list in the writeup, updated now.
lawrencedudley
commented
3 months ago Sitting on the sidelines here but just wanted to say thanks for working on fixing this!
mkz212
commented
3 months ago @heldchen
I already stuck at the first point. I have error 400. I did everything as you stated. I only deleted these 3 lines because I don't know where to get this data?
"_csrf": "cIf...WjI",
"state": "hKF...KeA",
"nonce": "dW3...OSg",I think that these values also can be hardcoded? They are definitely not linked to the account because you are not logging in yet, only after this step are the username and password sent and you have these values before first post.
headers:
body:
heldchen
commented
3 months ago those are values found in the requests & login form before submitting it - and from google recaptcha which potentially is an issue as those are dynamically injected using javascript and then server-side verified. I think the more promising way to try first is to use the published oauth endpoints using an oauth client instead of their legacy (it says "deprecated" in the response) login form.
pre-login the following requests are done (I'll update the writeup a bit later with all details):
the very first request when starting the app's login process is:
(so all these values are somehow known to the app so most likely are either irrelevant or can be recreated/regenerated through code)
this then redirects to:
which returns the login html form.
danielcherubini
commented
3 months ago @mkz212 those values can't be hard coded, they are specifically there to show that the flow is being done correctly
CSRF - Cross Site Request Forgery token, it's something the server gives to you to use in your requests. State - could be anything, but commonly used to pass state between each request. Nonce - a single use token
mkz212
commented
3 months ago Looking from the end.. From last step..
We need X-Client-Id and X-User-Authorization-V2 (token).
Getting X-Client-Id having a token is a simple task.
To get first token, we need:
Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9, (hardcoded)client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx, (hardcoded, different for every App Store app version)"code": WE NEED THIS! This code is probably different for everyone because we provide username and password in previous step and code is only value that we get from there.?"code_verifier":WE NEED THIS?Getting refresh token is also easy when we get first token, cause we get there"refresh_token": and
"id_token":
danielcherubini
commented
3 months ago Looking from the end.. From last step..
- We need
X-Client-IdandX-User-Authorization-V2(token).- Getting
X-Client-Idhaving a token is a simple task.- To get first token, we need:
- header:
Auth0-Client: eyJuYW1lIjoiYXV0aDAuanMtdWxwIiwidmVyc2lvbiI6IjkuMjMuMiJ9,(hardcoded)- data:
client_id": "Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx,(hardcoded)"code":WE NEED THIS!"code_verifier":WE NEED THIS?
- Getting refresh token is also easy when we get first token, cause we get there
"refresh_token":and"id_token":
Code and code_verifier means it's using the PKCE flow.
Check step 1 and 2. You create those two values
danielcherubini
commented
3 months ago As i discussed before, it seems that Panasonic is using auth0 as their authentication provider. So it might be best to just use the auth0 SDK. This will also provide the auth0-client
mkz212
commented
3 months ago @danielcherubini @heldchen In my opinion, it all comes down to getting code and code_verifier values. With these values, it's probably easy to move on.
danielcherubini
commented
3 months ago @danielcherubini @heldchen In my opinion, it all comes down to getting
codeandcode_verifiervalues. With these values, it's probably easy to move on.
Check my reply to you. It's in there.
danielcherubini
commented
3 months ago @mkz212 you're using postman i see, so you're going to need a pre-request script here to generate those values
code
// Dependency: Node.js crypto module
// https://nodejs.org/api/crypto.html#crypto_crypto
function base64URLEncode(str) {
return str.toString('base64')
.replace(/\+/g, '-')
.replace(/\//g, '_')
.replace(/=/g, '');
}
var code = base64URLEncode(crypto.randomBytes(32));code_verifier
// Dependency: Node.js crypto module
// https://nodejs.org/api/crypto.html#crypto_crypto
function sha256(buffer) {
return crypto.createHash('sha256').update(buffer).digest();
}
var code_verifier = base64URLEncode(sha256(verifier));You're also probably going to have to have code_challenge_method and have that value be S256 because you're using Sha256 in the verifier
danielcherubini
commented
3 months ago in python land this looks like this https://github.com/gateley-auth0/CLI-PKCE/blob/master/login.py
mkz212
commented
3 months ago @danielcherubini please see that we need 'code' value to generate token. And we get code in previous step.
danielcherubini
commented
3 months ago wait, i remembered wrong, @mkz212 check the python script above, in the /login call you should send in the payload response_type: "code"
that response will reply with a value for code .. that's what the value of code is
danielcherubini
commented
3 months ago it's all in the documentation
check the numbered steps there.
danielcherubini
commented
3 months ago OK i have a few hours, i'll make the changes and open a PR
mkz212
commented
3 months ago OK i have a few hours, i'll make the changes and open a PR
Maybe first, together with @heldchen , refine step by step workflows to get these data? So every dev can update plugins.
danielcherubini
commented
3 months ago those are values found in the requests & login form before submitting it - and from google recaptcha which potentially is an issue as those are dynamically injected using javascript and then server-side verified. I think the more promising way to try first is to use the published oauth endpoints using an oauth client instead of their legacy (it says "deprecated" in the response) login form.
pre-login the following requests are done (I'll update the writeup a bit later with all details):
the very first request when starting the app's login process is:
(so all these values are somehow known to the app so most likely are either irrelevant or can be recreated/regenerated through code)
this then redirects to:
which returns the login html form.
@heldchen Thinking about this, i actually think we will have to do this login flow, where we open the URL, and get the values to continue. perhaps even using the form.
nelsongraca
commented
3 months ago @danielcherubini that is how oauth is supposed to be implemented, one issue here is that we can't control the redirect url, since the server only accepts authorized ones, one way I saw this being worked around was spawning a browser to open the oauth page and controlling it in order to intercept the redirect and handle it.
Best way I see is to use the android redirect url which is not valid in a computer, user gets redirected and can then copy paste the url to be processed
danielcherubini
commented
3 months ago Best way I see is to use the android redirect url which is not valid in a computer, user gets redirected and can then copy paste the url to be processed
Yeah, they will have configured redirect_url serverside, so if it doesn't match, it fails.. that i know. We CAN intercept this, which is the whole reason PKCE exists. We would need to intercept the exact redirect_url they have, with the panasonic-iot-cfc:// as well.
That's probably not going to work with this library as it currently stands. It would need heavy refactoring.
The flow would then be
/authorize urldanielcherubini
commented
3 months ago Personally, my esp32 arrives tomorrow. I will be going down the hardware local route from here on in.
Wish everyone luck
heldchen
commented
3 months ago Code and code_verifier means it's using the PKCE flow.
Check step 1 and 2. You create those two values
https://curity.io/resources/learn/python-openid-connect-client/ has an example how to use the code_verifier & code_challenge. with this, a login browser url can be constructed, which the user can use to login and get the code value:
code_verifier string: code_verifier = tools.generate_random_string(100)code_challenge = tools.base64_urlencode(hashlib.sha256(code_verifier).digest())https://authglb.digital.panasonic.com/authorize url & open it in a browser (all url params are now known values)code parameter in the https://authglb.digital.panasonic.com/authorize/resume responsenelsongraca
commented
3 months ago An option is how some unofficial Tesla API clients worked, they did the whole login flow, including getting the form, parse it, and send the responses, 2fa code included.
heldchen
commented
3 months ago the issue is not the form parsing, the issue is the google recaptcha. this nonce is client-side created by recaptcha and then verified server-side. this is currently the only blocker. the rest can all be reworked quite easily: I managed to create a code_verifier (needs to be unique and is one-time use only) and its code_challenge that survived the entire login worklow and then let me - with the code received at the end of the login process - successfully get the token.
import hashlib, base64, random, string
code_verifier=''.join(random.choice(string.ascii_lowercase + string.ascii_uppercase + string.digits) for _ in range(43))
code_challenge=base64.urlsafe_b64encode(hashlib.sha256(code_verifier.encode('utf-8')).digest()).split('='.encode('utf-8'))[0].decode('utf-8')
state=''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(20))
print('code_verifier: ' + str(code_verifier))
print('code_challenge: ' + str(code_challenge))
print('state: ' + state)
print('authorize url: https://authglb.digital.panasonic.com/authorize?scope=openid%20offline_access%20comfortcloud.control%20a2w.control&audience=https%3A%2F%2Fdigital.panasonic.com%2FXmy6xIYIitMxngjB2rHvlm6HSDNnaMJx%2Fapi%2Fv1%2F&response_type=code&code_challenge=' + code_challenge + '&code_challenge_method=S256&auth0Client=eyJuYW1lIjoiQXV0aDAuQW5kcm9pZCIsImVudiI6eyJhbmRyb2lkIjoiMzAifSwidmVyc2lvbiI6IjIuOS4zIn0%3D&client_id=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&redirect_uri=panasonic-iot-cfc%3A%2F%2Fauthglb.digital.panasonic.com%2Fandroid%2Fcom.panasonic.ACCsmart%2Fcallback&state=' + state)open the url in a browser. complete the login. in the final url the code can be found, which then can be used to grab the token:
curl 'https://authglb.digital.panasonic.com/oauth/token' \
-H 'auth0-client: eyJuYW1lIjoiQXV0aDAuQW5kcm9pZCIsImVudiI6eyJhbmRyb2lkIjoiMzAifSwidmVyc2lvbiI6IjIuOS4zIn0' \
-H 'content-type: application/json' \
-H 'user-agent: okhttp/4.10.0' \
--data-raw $'{"scope": "openid", "client_id":"Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx","grant_type":"authorization_code","code":"<CODE>","redirect_uri":"panasonic-iot-cfc://authglb.digital.panasonic.com/android/com.panasonic.ACCsmart/callback","code_verifier":"<CODE_VERIFIER>"}'(replace <CODE> & <CODE_VERIFIER>)
once we have a token (and its refresh token) the refresh token can be used to get a new auth token "forever", so this cumbersome process is only required once (unless you logout, probably)
heldchen
commented
3 months ago based on https://github.com/cjaliaga/aioaquarea/blob/9348dd47b2c13163c45748ed513853646b5b27dd/aioaquarea/core.py#L362 it seems that the google recaptcha is not enforced so the login step could be automated too
little-quokka
commented
3 months ago Demo for the getting-token steps: https://github.com/sockless-coding/panasonic_cc/issues/191#issuecomment-2177684277
helicalchris
commented
3 months ago Personally, my esp32 arrives tomorrow. I will be going down the hardware local route from here on in.
Wish everyone luck
How are you doing it locally using an ESP32? Can you link to how that integrates, as my system has 1 wall controller per unit (no IR)
Not the right place to discuss this at length, but thought others might find it useful too
danielcherubini
commented
3 months ago lawrencedudley
commented
3 months ago I'm gonna have a go at cobbling a local controller together, it looks pretty straightforward.
JHCD
commented
3 months ago Thanks a lot @danielcherubini and @little-quokka for the fast solution: https://github.com/lostfields/python-panasonic-comfort-cloud/pull/94
little-quokka
commented
3 months ago For the latest progress on the pull request, see here: https://github.com/lostfields/python-panasonic-comfort-cloud/pull/94
mcbrain78
commented
2 months ago Hi all,
i pulled the latest from github but it fails with a 403. I played a bit around with the headers and tried to create a token manually (since the automatic switch over seemed to be broken) which at least got me to 400 / "bad request".
Is the authentication still broken or am I doing something wrong?
Best, Markus
heldchen
commented
2 months ago you need to logout & login in their app probably.
mcbrain78
commented
2 months ago Unfortunately, that didnt fix it. I pulled the repo from github, logged in and out of the app and run (mytoken is a filename for a file which doesnt exist yet):
pcomfortcloud.py -t mytoken my_email my_password list
I get the result: File "C:\Users\mcbrain\PycharmProjects\python-panasonic-comfort-cloud\pcomfortcloud\session.py", line 99, in login self._create_token() File "C:\Users\mcbrain\PycharmProjects\python-panasonic-comfort-cloud\pcomfortcloud\session.py", line 135, in _create_token raise ResponseError(response.status_code, response.text) pcomfortcloud.session.ResponseError: Invalid response, status code: 403 - Data: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
heldchen
commented
2 months ago token logins are not supported anymore. panasonic switched to oauth bearer tokens. there's a pull request for this here, you will need to use this until it has been merged. https://github.com/lostfields/python-panasonic-comfort-cloud/pull/94
mcbrain78
commented
2 months ago thanks!
Hi @lostfields
The library is no longer able to authenticate after a recent update on the Panasonic Comfort Cloud side.
Logs:
This has also been raised with the HA custom integration here where this library is used.