Closed fkhera closed 3 years ago
From what i've seen so far, removing all the custom headers except User Agent is working so far for us. From what I heard their WAF is messed up and that is causing issues.
Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ?
On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata notifications@github.com wrote:
From what i've seen so far, removing all the custom headers except User Agent is working so far for us. From what I heard their WAF is messed up and that is causing issues.
β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782700248, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA .
Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ? β¦ On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata @.**> wrote: From what i've seen so far, removing all the custom headers except User Agent* is working so far for us. From what I heard their WAF is messed up and that is causing issues. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA .
Could be, not sure as of yet. Would also recommend adding auto retries if a timeout does happen. So far I am seeing greater amounts of timeouts than usual.
Our scripts worked perfectly up until this morning I am hoping Tesla servers are just messed up.
On Sat, Feb 20, 2021 at 8:42 AM Hemang Bhogayata notifications@github.com wrote:
Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ? β¦ <#m-5363665892503988508> On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata @.**> wrote: From what i've seen so far, removing all the custom headers except User Agent* is working so far for us. From what I heard their WAF is messed up and that is causing issues. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment) https://github.com/timdorr/tesla-api/issues/321#issuecomment-782700248>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA .
Could be, not sure as of yet. Would also recommend adding auto retries if a timeout does happen. So far I am seeing greater amounts of timeouts than usual.
β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782700945, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYAHOFYONJF3VDZX7T3S77J4TANCNFSM4X57EEOA .
Our scripts worked perfectly up until this morning I am hoping Tesla servers are just messed up. On Sat, Feb 20, 2021 at 8:42 AM Hemang Bhogayata notifications@github.com wrote: β¦ Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ? β¦ <#m-5363665892503988508> On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata @.**> wrote: From what i've seen so far, removing all the custom headers except User Agent* is working so far for us. From what I heard their WAF is messed up and that is causing issues. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment) <#321 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA . Could be, not sure as of yet. Would also recommend adding auto retries if a timeout does happen. So far I am seeing greater amounts of timeouts than usual. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYAHOFYONJF3VDZX7T3S77J4TANCNFSM4X57EEOA .
Yeah, I think I started seeing the issue pop around 6-7 hours ago.
I was checking my code the only header we use is User Agent : https://github.com/fkhera/powerwallCloud/blob/abd0daa069888e68b346201464ec5dfcda771144/powerwallBackup.py#L138
On Sat, Feb 20, 2021 at 8:51 AM Hemang Bhogayata notifications@github.com wrote:
Our scripts worked perfectly up until this morning I am hoping Tesla servers are just messed up. On Sat, Feb 20, 2021 at 8:42 AM Hemang Bhogayata notifications@github.com wrote: β¦ <#m3377987060617303087> Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ? β¦ <#m-5363665892503988508> On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata @.**> wrote: From what i've seen so far, removing all the custom headers except User Agent* is working so far for us. From what I heard their WAF is messed up and that is causing issues. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 https://github.com/timdorr/tesla-api/issues/321 (comment) <#321 (comment) https://github.com/timdorr/tesla-api/issues/321#issuecomment-782700248>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA . Could be, not sure as of yet. Would also recommend adding auto retries if a timeout does happen. So far I am seeing greater amounts of timeouts than usual. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment) https://github.com/timdorr/tesla-api/issues/321#issuecomment-782700945>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYAHOFYONJF3VDZX7T3S77J4TANCNFSM4X57EEOA .
Yeah, I think I started seeing the issue pop around 6-7 hours ago.
β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782702294, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYBYK33MWKGX3QOR5C3S77K6LANCNFSM4X57EEOA .
I was checking my code the only header we use is User Agent : https://github.com/fkhera/powerwallCloud/blob/abd0daa069888e68b346201464ec5dfcda771144/powerwallBackup.py#L138 On Sat, Feb 20, 2021 at 8:51 AM Hemang Bhogayata notifications@github.com wrote: β¦ Our scripts worked perfectly up until this morning I am hoping Tesla servers are just messed up. On Sat, Feb 20, 2021 at 8:42 AM Hemang Bhogayata @. wrote: β¦ <#m3377987060617303087> Is that a temporary thing or should we just do that going forward removing all custom headers except usage agent ? β¦ <#m-5363665892503988508> On Sat, Feb 20, 2021 at 8:38 AM Hemang Bhogayata @.> wrote: From what i've seen so far, removing all the custom headers except User Agent is working so far for us. From what I heard their WAF is messed up and that is causing issues. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 <#321> (comment) <#321 (comment) <#321 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYG6NDIW5ZUERHU3USDS77JN7ANCNFSM4X57EEOA . Could be, not sure as of yet. Would also recommend adding auto retries if a timeout does happen. So far I am seeing greater amounts of timeouts than usual. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment) <#321 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYAHOFYONJF3VDZX7T3S77J4TANCNFSM4X57EEOA . Yeah, I think I started seeing the issue pop around 6-7 hours ago. β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#321 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYBYK33MWKGX3QOR5C3S77K6LANCNFSM4X57EEOA .
What is the exact error or response you are getting from Tesla API ? Since changing the headers, for me it seems like a hit or miss. We are also seeing various issues like API giving errors of Incorrect Password even though the password is correct. To add, I am also seeing a lot of MFA accounts being reported back as Non-Mfa by the API. Something seems to be up with the API
Okay so really all I am seeing is the 2nd post request that embeds credentials, crsf token process authorize all that I hanging. Not receiving a response back:
headers = {
'User-Agent' : 'PowerwallDarwinManager'
}
resp = session.get(auth_url, headers=headers)
print resp
csrf = re.search(r'name="_csrf".+value="([^"]+)"', resp.text).group(1)
transaction_id = re.search(r'name="transaction_id".+value="([^"]+)"', resp.text).group(1)
data = {
"_csrf": csrf,
"_phase": "authenticate",
"_process": "1",
"transaction_id": transaction_id,
"cancel": "",
"identity": self.email,
"credential": self.password,
}
print data
print "Opening session with login"
# Important to say redirects false cause this will result in 302 and need to see next data
resp = session.post(auth_url, headers=headers, data=data, allow_redirects=False)
Seeing the same, with similiar code base. Does very occasionally go through.
On one occasion got as reply from "https://auth.tesla.com/oauth2/v3/authorize",
<script type="text/javascript">
(function(){
window["bobcmn"] = "101110111110102000000042000000052000000062000000012a83e006b200000096200000000200000002300000000300000000300000006/TSPD/300000008TSPD_10130000000cTSPD_101_DID300000005https3000000b008ce580718ab20005bb03f1c0ba30570c771e2169cfb14a98e182b4c2338eae309bdfdd18aa36e590897e1fd990a2800ba3a0e7d849a918b9a6ed53ada5855b2de242cb70aa8e5a3de7397492b5301f459744d4489a917bd300000002TS200000000200000000";
No _csrf, or transaction_id...
This Reply seems to be F5 Networks javascript bot detection.. as detailed in API https://tesla-api.timdorr.com/api-basics/authentication#get-https-auth-tesla-com-oauth-2-v-3-authorize
May be temporary block, because of number of requests or may be new security...
Moving to using refresh_token, and will wait it out and see.
This can be resolved by essentially removing all headers from any requests, besides the bearer token. Tesla "broke" their WAF and any requests with the headers "x-tesla-user-agent" or "X-Requested-With" will result in a timeout.
Related: https://github.com/timdorr/tesla-api/discussions/320
Thanks - seems all headers need to go...
https://github.com/enode-engineering/tesla-oauth2/commit/ab40d7545781de7ad93d4754bfe258cd7540e6c2
Is this permanent ?
On Sat, Feb 20, 2021 at 5:17 PM Ghawken notifications@github.com wrote:
Thanks - seems all headers need to go...
enode-engineering/tesla-oauth2@ab40d75 https://github.com/enode-engineering/tesla-oauth2/commit/ab40d7545781de7ad93d4754bfe258cd7540e6c2
β You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782770044, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYDS5JLBUH6GUZFUZCDTABGKHANCNFSM4X57EEOA .
@fkhera Nobody knows as there is no communication with Tesla's team. I would not expect this to be permanent and you may need to adjust any functions again if further changes occur.
There seem to be some other changes too. My MFA calls were all failing because they needed extra headers, including Accept
and apparently Referrer
. I got past that, but now I'm getting no response on the call to https://auth.tesla.com/oauth2/v3/token
. I'll keep trying stuff but if anyone has auth working end to end, let me know if you have any further tips.
@tomhollander The code shared by @Ghawken above (enode-engineering/tesla-oauth2@ab40d75) shows auth (including MFA) working end to end and I use this personally on my services.
There may be small differences from codebase to codebase, but there shouldn't be a single request that requires headers beyond the request that hits https://owner-api.teslamotors.com/oauth/token
, however, this is technically just needed for the creation of the Owners API tokens and not needed for the SSO process.
headers["authorization"] = "bearer " + access_token
payload = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"client_id": CLIENT_ID,
}
resp = session.post("https://owner-api.teslamotors.com/oauth/token", headers=headers, json=payload)
@tomhollander The code shared by @Ghawken above (enode-engineering/tesla-oauth2@ab40d75) shows auth (including MFA) working end to end and I use this personally on my services.
Hmm, I just tried this implementation and it's not working for me either, although I'm not sure it's the same problem. My code (https://github.com/tomhollander/TeslaAuth) was working fine (including with MFA) fine until today, but I clearly need more changes to get it past their latest update.
This can be resolved by essentially removing all headers from any requests, besides the bearer token. Tesla "broke" their WAF and any requests with the headers "x-tesla-user-agent" or "X-Requested-With" will result in a timeout.
Related: #320
Ok i just got my scripts back up and running again. Removing headers approach seems to be the way. Earlier I kept only User Agent and as a result saw weird isssues. Upon removing all headers, my script is working like before on local as well as on AWS. Thanks for you help π
Today itβs getting stuck further down the chain. I can verify itβs non Mfa which is correct in my case : Hanging now getting access token so appears Tesla is making progress on fixing waf ?
https://github.com/fkhera/powerwallCloud/blob/master/powerwallBackup.py#L252
On Sun, Feb 21, 2021 at 2:13 AM Hemang Bhogayata notifications@github.com wrote:
This can be resolved by essentially removing all headers from any requests, besides the bearer token. Tesla "broke" their WAF and any requests with the headers "x-tesla-user-agent" or "X-Requested-With" will result in a timeout.
Related: #320 https://github.com/timdorr/tesla-api/discussions/320
Ok i just got my scripts back up and running again. Removing headers approach seems to be the way. Earlier I kept only User Agent and as a result saw weird isssues. Upon removing all headers, my script is working like before on local as well as on AWS. Thanks for you help π
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782824762, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYF54HWFFXGSM2ZMXZDTADFDVANCNFSM4X57EEOA .
A few calls working now looks like Tesla almost fixed it .
On Sun, Feb 21, 2021 at 8:26 AM Farooq Khera fkhera@asu.edu wrote:
Today itβs getting stuck further down the chain. I can verify itβs non Mfa which is correct in my case : Hanging now getting access token so appears Tesla is making progress on fixing waf ?
https://github.com/fkhera/powerwallCloud/blob/master/powerwallBackup.py#L252
On Sun, Feb 21, 2021 at 2:13 AM Hemang Bhogayata notifications@github.com wrote:
This can be resolved by essentially removing all headers from any requests, besides the bearer token. Tesla "broke" their WAF and any requests with the headers "x-tesla-user-agent" or "X-Requested-With" will result in a timeout.
Related: #320 https://github.com/timdorr/tesla-api/discussions/320
Ok i just got my scripts back up and running again. Removing headers approach seems to be the way. Earlier I kept only User Agent and as a result saw weird isssues. Upon removing all headers, my script is working like before on local as well as on AWS. Thanks for you help π
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-782824762, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYF54HWFFXGSM2ZMXZDTADFDVANCNFSM4X57EEOA .
Started having this problem this morning. It worked once randomly then stopped working and seems to timeout when trying to exchange the auth code for the bearer token. I'm not quite understanding the part about removing all the headers during the authentication process as the only header I'm adding is the Cookie session-id for the auth code request, which I tried to remove but still did not work.
I've confirmed my library code works as-is, but the SSO service does appear to be heavily rate limited. Somewhere on the order of 2-4 rpm at max. You might be able to bust the WAF by changing headers randomly, but no guarantees there and suspicious behavior may also put you at risk at stronger blocks.
Anyways, since this is a Tesla issue, not an issue with these docs, I'm going to close this out. I would leave Tesla-side issues to the discussions, since there's not going to be any resolution here. It'll get fixed whenever the person with the pager at Tesla fixes it. π€·
@timdorr I've finally got my library working again, and there are definitely some changes to the API behaviour that need to be documented. Specifically for the call in step 3, some specific headers seem to be needed:
Accept: application/json
Accept-Encoding: gzip, deflate
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Looks like something new is going on. Anyone else not able to get anything through right now?
New WAF configuration was deployed at 2pm PST. Not sure what the new rules are, but it definitely broke the new implementations as well.
I am also broken do we need to add the user agent back ??
On Thu, Feb 25, 2021 at 4:50 PM James Gragg notifications@github.com wrote:
New WAF configuration was deployed at 2pm PST. Not sure what the new rules are, but it definitely broke the new implementations as well.
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786309354, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYGQEJF5DXM5PDOKVL3TA3O2ZANCNFSM4X57EEOA .
@fkhera I tried that as well as using multiple proxies with no luck.
Yup doesnβt matter what header is still hanging.
On Thu, Feb 25, 2021 at 4:55 PM jaggerc notifications@github.com wrote:
@fkhera https://github.com/fkhera I tried that as well as using multiple proxies with no luck.
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786311100, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYABELGPTCSL4MKVKI3TA3POFANCNFSM4X57EEOA .
Also experiencing this. Investigating.
After running mitmproxy, it looks like there are required querystring parameters. We now have audience
, locale
, and prompt
as new ones. With those and the previous headers, I'm getting successful auth flow responses again.
The querystring parameters are included in the GET for the login form, but not the POST. Just adding those querystring parameters to the GET and re-adding headers for all GET and POSTs seems to work. There are still a lot of timeouts, but retrying does work.
Can you comment on the exact parameters on the initial call for the session login? My code is just getting hung at login:
Trying:
On Thu, Feb 25, 2021 at 6:44 PM can-bus-burger notifications@github.com wrote:
The querystring parameters are included in the GET for the login form, but not the POST. Just adding those querystring parameters to the GET and re-adding headers for all GET and POSTs seems to work. There are still a lot of timeouts, but retrying does work.
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786351169, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYFIYSICXMG3CUVZSXDTA34GFANCNFSM4X57EEOA .
Other than audience param being blank, that looks pretty much the same as mine.
Other than audience param being blank, that looks pretty much the same as mine.
Aww okay setting audience=""
, gets me to v3 token call, but hanging there.
They are definitely throttling by your header id, so may need to randomize user_agent now.
On Thu, Feb 25, 2021 at 6:51 PM can-bus-burger notifications@github.com wrote:
Other than audience param being blank, that looks pretty much the same as mine.
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786353774, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYBCCZVUD26PDVVLPE3TA35DDANCNFSM4X57EEOA .
They are definitely throttling by your header id, so may need to randomize user_agent now.
Did you get it working just by adding audience = "" ?
so adding the empty audience gets me to : resp = session.post("https://auth.tesla.com/oauth2/v3/token", headers=headers, json=payload), however here it is stalling, so either this call needs more Query params, different payload, is it possible to run the mini proxy when its doing token to get its new variables/payload?
Where payload is:
payload = {
"grant_type": "authorization_code",
"client_id": "ownerapi",
"code_verifier": self.rand_str(108),
"code": code,
"redirect_uri": "https://auth.tesla.com/void/callback",
}
@timdorr is it possible to re-open this issue?
@fkhera is the code verifier being regenerated or the one you used to make the code_challenge?
It should be the same one used in challenge of course
On Thu, Feb 25, 2021 at 7:51 PM can-bus-burger notifications@github.com wrote:
@fkhera https://github.com/fkhera is the code verifier being regenerated or the one you used to make the code_challenge?
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786374357, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYHA2SAPFHAQ42UZBA3TA4EEZANCNFSM4X57EEOA .
Just wanted to share that we have updated our public script of the tesla oauth documentation . We have decided to just run a headless chrome web driver to reliably get cookies, csrf and transaction_id. Don't want to deal with these changes any more :)
How do you exchange for the v3 token ?
On Fri, Feb 26, 2021 at 3:18 AM Nikolai Johan Heum notifications@github.com wrote:
Just wanted to share that we have updated our public script of the tesla oauth documentation https://github.com/enode-engineering/tesla-oauth2. We have decided to just run a headless chrome web driver to reliably get cookies, csrf and transaction_id. Don't want to deal with these changes any more :)
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786553440, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYBZY5GT4DFMXNWGH7TTA5YNXANCNFSM4X57EEOA .
My old header less code worked again strange ...
On Fri, Feb 26, 2021 at 5:49 AM Farooq Khera fkhera@asu.edu wrote:
How do you exchange for the v3 token ?
On Fri, Feb 26, 2021 at 3:18 AM Nikolai Johan Heum < notifications@github.com> wrote:
Just wanted to share that we have updated our public script of the tesla oauth documentation https://github.com/enode-engineering/tesla-oauth2. We have decided to just run a headless chrome web driver to reliably get cookies, csrf and transaction_id. Don't want to deal with these changes any more :)
β You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/321#issuecomment-786553440, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4JYBZY5GT4DFMXNWGH7TTA5YNXANCNFSM4X57EEOA .
Indeed, My old scripts without any headers are functional again and weirdly enough taking lot less time to authenticate than before.
Hello: Our Python scripts failed This morning is Tesla auth down anyone know ?
https://auth.tesla.com/oauth2/v3/authorize?code_challenge=cNJmb_XD0HY8jj2SgTRN2s18Xy1C8BHKhhAbzJna_AU&client_id=ownerapi&state=tesla_exporter&redirect_uri=https%3A%2F%2Fauth.tesla.com%2Fvoid%2Fcallback&response_type=code&code_challenge_method=S256&scope=openid+email+offline_access
when I tried manual login I get 404 error need to deep dive more . Anyone else see their scripts go down ?