BUG: tesla_api.py config update files during upgrade

[mjpcomp]

During the latest update, I'm guessing after the new wake method was introduced, noticed this:

The vehicle has not been staying awake since the transition to the new method was made in the code (didn't see any documentation, other than the commit notes, on how to make sure the new methods are used).

[mjpcomp]

Noticed that the /mutable/tesla_api.json file was completely empty... deleted it, re-ran the setup upgrade command... now: I just see my e-mail address in there, but, no token/id/vehicle_id...

[marcone]

Does /mutable/cache.json exist?

[mjpcomp]

Does /mutable/cache.json exist?

No sign of /mutable/cache.json at all..

Here's what I see (there are some other files/folders which I've added over the year(s)):

The OLDtesla_api.json was a backup from the beginning of 2020, so, ignore that...

[marcone]

Try deleting tesla_api.json again, then running tesla_api.py --email <youremail> --password <yourpassword> list_vehicles manually. Might show what goes wrong. I suspect Tesla is prompting for a captcha, which isn't supported.

[mjpcomp]

D'oh - CAPTCHA required:

"ValueError: Credentials rejected. Captcha is required. Captcha does not match"

Any way to set it so if the JSON is invalid (missing data), that we don't keep seeing the: "failed to contact car, retrying" attempt every 13 seconds or so?

Maybe if the user has notifications enabled, send out one that mentioned that a vehicle connection isn't possible at this time, and to comment out the connection option in the configuration?

But, I take it if we get the tokens manually, we are okay to proceed?

[mjpcomp]

Hmm, it does look like a fix was implemented in TeslaPy for this: https://github.com/tdorssers/TeslaPy/issues/29

[marcone]

That still requires user interaction to solve the captcha though (TeslaPy will display the captcha image and you have to type what it says), so won't work for teslausb headless setup.

[rmyadsk]

Is there a way to integrate a token obtained outside of the native tesla_api.py? Since Tesla implemented the Captcha, I've been looking for a package that would generate one, and I think I may have a working setup using this fetcher. Using that I'm able to generate an Access Token, an Owner-Api Token, and a Refresh Token. Plugging the Owner-Api Token into /mutable/tesla_api.json and running tesla_api.py --email <youremail> --password <yourpassword> list_vehicles results in a nicely populated JSON structure with my vehicles, but after running setup-teslausb upgrade following that, I still have a setup that goes to sleep.

Following that, no combination of id and vehicle_id values that I try to plug into /mutable/tesla_api.json seem to make any difference, and archiveloop.log always has a not keeping car awake. line early on after boot - awake_start seems to want a /mutable/cache.json file that doesn't exist.

[marcone]

Teslausb uses teslapy so if you can authenticate manually with that, you could use the generated "cache.json" with teslausb.

[rmyadsk]

Thanks @marcone, I think I have it. My generated cache.json token is good for the next 45 days and appears to work on both vehicles. If more than one Tesla is registered to an account, it is beneficial (and likely necessary) to run tesla_api.py --email <youremail> --password <yourpassword> --vin [target_VIN] get_vehicle_data after the access_token field has been populated in tesla_api.json (it appears with the same key name in cache.json). That will populate tesla_api.json with remaining (preferred and correct?) syntax, which appears to be quoted for the value of "id" and unquoted for the value of "vehicle_id". Then a /root/bin/setup-teslausb upgrade should pick up any changes.

[marcone]

I'm not sure, but don't think you need to provide your email and password once you have a token. If you have more than one car on the account you do need to specify which car to use, which can be done either by VIN or by name.

[basalblas]

I got it working again by manually adding my TESLA_ACCESS_TOKEN to /mutable/tesla_api.json, running /root/bin/tesla_api.py --email list_vehicles and copying /mutable/tesla_api.json to /mutable/cache.json. After that, mounting my CAM and MUSIC partitions on the car was no longer an issue anymore.

You can get a TESLA_ACCESS_TOKEN by AuthAppForTesla on an IOS device for example, but there are other ways as well. We'll see if the token will be auto renewed after 45 days (when it will expire).

[marcone]

... copying /mutable/tesla_api.json to /mutable/cache.json.

tesla_api.json is used by tesla_api.py, whereas cache.json is used by the teslapy library. They contain different data, and copying one to the other makes little sense. In fact, I would expect what you described to not work at all, at least not on the current version of teslausb. It might work if your teslausb is older than June 25th (in which case you don't need cache.json, and you'll need to manually get a new token once the current token expires)

[basalblas]

... copying /mutable/tesla_api.json to /mutable/cache.json.

tesla_api.json is used by tesla_api.py, whereas cache.json is used by the teslapy library. They contain different data, and copying one to the other makes little sense. In fact, I would expect what you described to not work at all, at least not on the current version of teslausb. It might work if your teslausb is older than June 25th (in which case you don't need cache.json, and you'll need to manually get a new token once the current token expires)

I've setup my teslausb using your latest image teslausb-20210815.zip. If I remove /mutable/cache.json the archiveloop.log shows not keeping car awake. If I copy /mutable/tesla_api.json to /mutable/cache.json the archiveloop.log shows Starting background task to keep car awake archiveloop.log

[rmyadsk]

If I copy /mutable/tesla_api.json to /mutable/cache.json the archiveloop.log shows Starting background task to keep car awake

I'm not sure this makes sense, at least not for new installs. tesla_api.json contains four fields: email, access_token, id, and vehicle_id. Ideally, tesla_api.py would read your account credentials from setup variables, authenticate against the Tesla API, and populate that file with car details, wherein a token would be generated, but remember Tesla has implemented a captcha requirement which blocks this auto-provisioning.

Instead, as a workaround for new installs, I found that running TeslaPy (the actual package, not the derived tesla_api.py included in teslausb) interactively - while answering the Captcha - can generate tokens. TeslaPy generates a cache.json which is a much different format than tesla_api.json - it describes specifically the token our tesla_api.py script should use to keep the car awake. Presumably, I'll need to regenerate this token every 45 days, unless Tesla decides to roll back the captcha.

[basalblas]

If I copy /mutable/tesla_api.json to /mutable/cache.json the archiveloop.log shows Starting background task to keep car awake

I'm not sure this makes sense, at least not for new installs. tesla_api.json contains four fields: email, access_token, id, and vehicle_id. Ideally, tesla_api.py would read your account credentials from setup variables, authenticate against the Tesla API, and populate that file with car details, wherein a token would be generated, but remember Tesla has implemented a captcha requirement which blocks this auto-provisioning.

Instead, as a workaround for new installs, I found that running TeslaPy (the actual package, not the derived tesla_api.py included in teslausb) interactively - while answering the Captcha - can generate tokens. TeslaPy generates a cache.json which is a much different format than tesla_api.json - it describes specifically the token our tesla_api.py script should use to keep the car awake. Presumably, I'll need to regenerate this token every 45 days, unless Tesla decides to roll back the captcha.

Be aware my /mutable/tesla_api.json is manually populated with the acces_token that I retrieved using the iOS app AuthAppForTesla. My teslausb_setup_variables.conf does not contain export TESLA_PASSWORD=teslapass. Only export TESLA_EMAIL= is defined. After adding the access-token to tesla_api.json I can run /root/bin/tesla_api.py --email <email address> list_vehicles which will show my Tesla's details, without asking for a captcha.

[basalblas]

If I copy /mutable/tesla_api.json to /mutable/cache.json the archiveloop.log shows Starting background task to keep car awake

I'm not sure this makes sense, at least not for new installs. tesla_api.json contains four fields: email, access_token, id, and vehicle_id. Ideally, tesla_api.py would read your account credentials from setup variables, authenticate against the Tesla API, and populate that file with car details, wherein a token would be generated, but remember Tesla has implemented a captcha requirement which blocks this auto-provisioning. Instead, as a workaround for new installs, I found that running TeslaPy (the actual package, not the derived tesla_api.py included in teslausb) interactively - while answering the Captcha - can generate tokens. TeslaPy generates a cache.json which is a much different format than tesla_api.json - it describes specifically the token our tesla_api.py script should use to keep the car awake. Presumably, I'll need to regenerate this token every 45 days, unless Tesla decides to roll back the captcha.

Be aware my /mutable/tesla_api.json is manually populated with the acces_token that I retrieved using the iOS app AuthAppForTesla. My teslausb_setup_variables.conf does not contain export TESLA_PASSWORD=teslapass. Only export TESLA_EMAIL= is defined. After adding the access-token to tesla_api.json I can run /root/bin/tesla_api.py --email <email address> list_vehicles which will show my Tesla's details, without asking for a captcha.

Correction to the above: My teslausb_setup_variables.conf does contain export TESLA_PASSWORD=teslapass, but this obviously is not my real Tesla account password. This causes tesla_api.py to get installed. I then manually add my access_token to the /mutable/tesla_api.json. After that tesla_api.py works fine, for example I can get my vehicle list, even if /mutable/cache.json does not yet exist. However, I found the awake scripts check for the cache.json file to exist and not be zero size. So I just created that file with some comments and the keep car awake function works. I think the only problem left is the access_token not getting renewed when it expires after 45 days, but we'll see.

[marcone]

Correction to the above: My teslausb_setup_variables.conf does contain export TESLA_PASSWORD=teslapass, but this obviously is not my real Tesla account password. This causes tesla_api.py to get installed. I then manually add my access_token to the /mutable/tesla_api.json. After that tesla_api.py works fine, for example I can get my vehicle list, even if /mutable/cache.json does not yet exist. However, I found the awake scripts check for the cache.json file to exist and not be zero size. So I just created that file with some comments and the keep car awake function works. I think the only problem left is the access_token not getting renewed when it expires after 45 days, but we'll see.

I still don't understand how that can work. awake_start calls tesla_api.py to talk to Tesla/car. Regardless of which of the two methods you picked to keep the car awake, tesla_api.py will call the _execute_request function, which in turn calls _rest_request, which in turn calls _get_api_token. _get_api_token uses the access token from the tesla_api.json file, but it also uses TeslaPy to check that the token is still valid, and TeslaPy only works with a properly-formed cache.json

[basalblas]

See my attached /mutable/tesla-api.json and /mutable/cache.json.

When I issue a /root/bin/tesla_api.py streaming_ping the response is <Response [101]> which looks fine to me?

I'm not sure TeslaPy still requires a properly-formed cache.json. Even if I temporarily rename cache.json to cache.json.old the above tesla_api.py command still works (as well as all other functions such as list_vehicles) tesla_api.json.txt cache.json.txt

[mnestor]

Teslapy needs a proper cache.json in order to use the access/refresh tokens to keep your auth updated. Without it auth will fail on the refresh_token call and you'll be unable to keep the car awake after the tokens expire.

That said though, teslausb does NOT need that cache.json file since it uses the written tokens from the setup file and the tesla_api.json files. When it calls teslapy to refresh tokens it will update the tesla_api.json file.

In teslapy2.0 they don't support headless setup. Kinda... You just have to give it a passcode_getter that returns your current passcode then you'll have tokens for it to update as needed. The easiest way I found to do this is run a few python commands myself.

You'll need your email, password and the current MFA token. Wait until it refreshes to put it in and run fetch_token() asap.

cd /mutable #so that the cache file is put in this location
python3  # start a python shell and paste the below

import teslapy # import teslapy 
t = teslapy.Tesla('email', 'password', passcode_getter=lambda :'MFATOKEN')
t.fetch_token()  # saves the token to cache.json
t.token # prints it out so now you get the values to put in your setup file

fetch_token() will write the cache.json file. Then just copy the new access_token and refresh_token to your setup file and re-run setup.

cache.json

root@teslausb:~# cat /mutable/cache.json
{
    "EMAIL": {
        "url": "https://auth.tesla.com/",
        "sso": {
            "access_token": "ACCESS_TOKEN",
            "refresh_token": "REFRESH_TOKEN",
            "id_token": "ID_TOKEN",
            "expires_in": 28800,
            "token_type": "Bearer",
            "expires_at": 234567890
        },
        "ownerapi": {
            "access_token": "ACCESS_TOKEN",
            "token_type": "bearer",
            "expires_in": 3888000,
            "refresh_token": "REFRESH_TOKEN",
            "created_at": 234567890
        }
    }
}

[withinfocus]

Looks like Tesla's made a breaking change with captchas and TeslaPy has also changed to compensate for it: https://github.com/tdorssers/TeslaPy/releases/tag/v2.0.0

[Vipercat]

@mnestor , I tried to create a cache.json file based on your instructions, but I receive an error while entering the command

_"t = teslapy.Tesla('my_email', 'my_password', passcode_getter=lambda :'mymfa')".

The error states: _Traceback (most recent call last): File "", line 1, in TypeError: init() got an unexpected keyword argument 'passcodegetter'

The error dissapears when I replace _passcodegetter=lambda with only lambda

However, when I then try to save the token in cache.json via _t.fetchtoken() I get a bunch of other errors:

_Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.7/dist-packages/teslapy/init.py", line 145, in fetch_token response = oauth.get(url) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 546, in get return self.request('GET', url, kwargs) File "/usr/local/lib/python3.7/dist-packages/requests_oauthlib/oauth2_session.py", line 516, in request method, url, headers=headers, data=data, kwargs File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 412, in send conn = self.get_connection(request.url, proxies) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 304, in get_connection proxy = prepend_scheme_if_needed(proxy, 'http') File "/usr/lib/python3/dist-packages/requests/utils.py", line 895, in prepend_scheme_if_needed scheme, netloc, path, params, query, fragment = urlparse(url, new_scheme) File "/usr/lib/python3.7/urllib/parse.py", line 367, in urlparse url, scheme, _coerce_result = _coerce_args(url, scheme) File "/usr/lib/python3.7/urllib/parse.py", line 120, in _coerceargs raise TypeError("Cannot mix str and non-str arguments") TypeError: Cannot mix str and non-str arguments

Any idea what I'm doing wrong or what's happening here?

[Vipercat]

Figured it out!

I had to change the variable for the location of cache.json in /usr/local/lib/python3.7/dist-packages/teslapy/ini.py (line 67) like this:

cache_file='/mutable/cache.json', cache_loader=None, cache_dumper=None):

[marcone]

Closing this since the way teslausb uses the Tesla API has changed again. See https://github.com/marcone/teslausb/commit/d91a63a054267057383612689aab443925fea40f