Missing sleep state, long wake times and errors in log with MCU2 updated cars

[markusdd]

Is there an existing issue for this?

• [X] I have searched the existing issues

What happened?

The car only switches between online and offline state, sleep does never occur. Also, the log is littered with weird API errors and the wakeup times of the car are far too frequent and long, leading to bad phantom drain. Also raised this in the discussion area: https://github.com/adriankumpf/teslamate/discussions/3083

Expected Behavior

the car goes to sleep (not offline), there are no API query errors in the log and the car actually sleeps when it is not being used or charged

Steps To Reproduce

if only I knew, but I can give my config:

• Streaming API is enabled
• apart fromt his weird behavior everything else works, drives etc are all recorded fine
• car works
• this is an 85D Model S with MCU2 upgrade, so no MCU1 energy save settings required/possible
• no other services are registered with my account except Teslamate

In the current example the below log output belongs to the last offline/online phase, error prints start happeneing at around the red arrow marker:

Relevant log output

see screen below

Screenshots

ok....and now after a long wake period we are suddenly offline....that is weird.

Additional data

No response

Type of installation

Manual

Version

v1.27.2

[micves]

please guys do not conflate the 2023.x issue and sentry mode with what was described here. It is unrelated.

Agreed that sentry is off topic here. But we might have more issues with 2023.38.x

I also have a 2015 pre facelift Model S upgraded to MCU2.

I used this patch in my local fork (on top of #3373) and I see no difference

@tpd-opitz, what Tesla version are you running? Is it 2023.38.x ? If so, did it work before you got that version?

It's a little disturbing that it might no longer be possible to detect real online vs. fake online :( Tesla could very well have changed more in the API than what is discovered in #3373 Maybe power in stream API is never nil anymore, which is the thing used to differ real from fake online at the moment

I don't have 2023.38.x yet, so I can't really test anything (yet)

If we have someone on this thread/fork with MCU2 retrofit and 2023.38.x, please speak up ;)

[DavidMansell]

I have 2023.38.x but don’t have an MCU2 upgrade car any more. :(

[tpd-opitz]

@micves my car is a Model-S 2015 before facelift on 2023.38.6 an has the MCU2 retrofit upgrade and I volunteer as test dummy. My car had long online times reported in teslamate even before the 2023.38.x update.

In my special case the car didn't sleep at all before the MCU upgrade (according to teslamate). After the MCU upgrade (in April) I began seeing "offline" times in teslamate.

As you can see in the data I provided I still have long online times reported in teslamate after applying this patch. Fortunately beside the long online times the short once became shorter...

[DumSkidderik]

Another 2015 with MCU2 owner here :) I pulled the docker image from https://hub.docker.com/r/leandereberhard/teslamate_mcu2_fix_micves to help test this, will report back if it changed anything.

[tpd-opitz]

@DumSkidderik I changed my docker-compose.yml to

$ head -6 docker-compose.yml 
version: "3"

services:
  teslamate:
    # image: teslamate/teslamate:latest
    image: leandereberhard/teslamate_mcu2_fix_micves:latest

But on pull I got:

$ docker-compose pull
Pulling teslamate ... error
Pulling database  ... done
Pulling grafana   ... done
Pulling mosquitto ... done

ERROR: for teslamate  manifest for leandereberhard/teslamate_mcu2_fix_micves:latest not found: manifest unknown: manifest unknown
ERROR: manifest for leandereberhard/teslamate_mcu2_fix_micves:latest not found: manifest unknown: manifest unknown

[micves]

@tpd-opitz Maybe the tag 'latest' doesn't work here. Try with leandereberhard/teslamate_mcu2_fix_micves:20231111

As you can see in the data I provided I still have long online times reported in teslamate after applying this patch. Fortunately beside the long online times the short once became shorter...

The long online can be normal. I also have those once in a while. It can be the 12V system being charged or similar secret stuff only known by Tesla :)

The small online every hour should not be there. Do you by chance have some other 3rd party software running having access to your Tesla?

[DumSkidderik]

I see a clear improvement:

[markusdd]

This is pretty much what I see since we've figured this out.

[micves]

@DumSkidderik Great and I hope you are on Tesla 2023.38.x so we can say this still works with recent API changes.

[DumSkidderik]

@DumSkidderik Great and I hope you are on Tesla 2023.38.x so we can say this still works with recent API changes.

2023.38.6

[USAFPride]

I’m on x.38.6 with 2015 S MCU2. I’m still using my own fork with this pull implemented and this is my last 24 hours. it will be this weekend before I can redo my setup to the new .org and merge in this pull, but the code should be identical. I do have the HA HACS addon. I’ll disable it and report back tomorrow.

[tpd-opitz]

@DumSkidderik

@tpd-opitz Maybe the tag 'latest' doesn't work here. Try with leandereberhard/teslamate_mcu2_fix_micves:20231111

This worked.

But I get an error when starting the image:

$ docker-compose logs -f  teslamate 
Attaching to teslamate_teslamate_1
teslamate_1  | exec /usr/bin/tini: exec format error
teslamate_1  | exec /usr/bin/tini: exec format error
teslamate_1  | exec /usr/bin/tini: exec format error
teslamate_1  | exec /usr/bin/tini: exec format error
teslamate_1  | exec /usr/bin/tini: exec format error
teslamate_teslamate_1 exited with code 1

As you can see in the data I provided I still have long online times reported in teslamate after applying this patch. Fortunately beside the long online times the short once became shorter...

The long online can be normal. I also have those once in a while. It can be the 12V system being charged or similar secret stuff only known by Tesla :)

The small online every hour should not be there. Do you by chance have some other 3rd party software running having access to your Tesla?

Well... I have openWB running checking SoC every hour while teslamate didn't work. Before the teslamate issue I had teslamate reporting the SoC to OWB via MQTT (and have it now again). And even then I had this short awake times without OWB polling the SoC:

But I'm disabling OWB access to my tesla account now.

[DumSkidderik]

I'm on 1.28dev edition using the docker image i mentioned, maybe you are still on 1.24?

[tpd-opitz]

@DumSkidderik quite likely since I don't know what you're talking about.. ;o)

I'm running the default teslamate install and only changed the teslamate image in the docker-compose.yml

[DumSkidderik]

@DumSkidderik quite likely since I don't know what you're talking about.. ;o)

I'm running the default teslamate install and only changed the teslamate image in the docker-compose.yml

Try this image:

[micves]

teslamate_1 | exec /usr/bin/tini: exec format error

@tpd-opitz this looks like you are on a different platform/architecture. Try this one that fits arm (raspberry pi): splidge0r/teslamate:latest

[DavidMansell]

Don't use that repo any more - it doesn't include the MCU2 fixes (I no longer have an MCU2 car, I've been trying to work around the 2023.38 issue instead).

[micves]

Ah okay, the update time and comment just still matched this issue :)

Let's hope this will be a thing soon: https://github.com/teslamate-org/teslamate/discussions/3416#discussioncomment-7565732

[USAFPride]

@micves, here is my states after 24 hours with no other 3rd party apps running. I’d say it is fixed.

[DumSkidderik]

Ah okay, the update time and comment just still matched this issue :)

Let's hope this will be a thing soon: #3416 (comment)

I'd like to mention that I'm back on the official docker image as the one i linked to seems to have it's own issues (tzdata errors), maybe I'll end up releasing my own with your fix :)

[tpd-opitz]

@ micves

teslamate_1 | exec /usr/bin/tini: exec format error

@tpd-opitz this looks like you are on a different platform/architecture.

I'm runnig teslamate on a raspberry 3

---

@DavidMansell which repo are you refering to?
this? splidge0r/teslamate:latest

[markusdd]

@micves do we have a repo state somewhere which fixes teslamate working at all on the newest software combined with our proposed fixes here?

[micves]

The PR #3262 is still open and I just synced it with official master yesterday. And now it might also work with the new automatic Docker build: https://github.com/micves?tab=packages

There are some issues and PR's if you search ghcr if you want to know more.

[tpd-opitz]

today my car updated to 2023.44.1 and for the first time ever I see my car sleeping in teslamate... :o)

[markusdd]

today my car updated to 2023.44.1 and for the first time ever I see my car sleeping in teslamate... :o)

Which version or fork are you running? Since the update I have no data capture at all anymore?

[micves]

Hmm, strange on so many levels. MCU2 retrofit in asleep... That should normally be offline. And the 1 hourly online is not suppose to happen on this fork.

The same with no data. The endpoint fix that was needed for 2023.38.x is merged in.

Info on Docker image or fork and some log output could be helpful

[tpd-opitz]

My teslamate ist this:

yes, I'm currently not on this fork...

[USAFPride]

@micves, I am on your branch and 2023.44.1 and now seeing "asleep"

[Dhtjf]

The PR #3262 is still open and I just synced it with official master yesterday. And now it might also work with the new automatic Docker build: https://github.com/micves?tab=packages

There are some issues and PR's if you search ghcr if you want to know more.

When attempting to pull from your repo, I'm getting a "manifest unknown" error message (running on windows docker desktop, utilizing Linux containers). Just upgraded the S to 2023.38.9 and would of course like to get the car where it'll sleep with teslamate running. I updated to the latest on the main teslamate branch without issues. Any advice?

[micves]

I just synced teslamate-org/master into my fork. It looked like there were some fixes to building the packages. I think new builds will be triggered and published soon on my fork under packages.

If the problem persists, I think we should open a new issue on it.

[micves]

Actually the packages on the fork might be wrong.

Try this one instead: https://github.com/teslamate-org/teslamate/pkgs/container/teslamate%2Fgrafana/155782587?tag=pr-3262

[Dhtjf]

Actually the packages on the fork might be wrong.

Try this one instead: https://github.com/teslamate-org/teslamate/pkgs/container/teslamate%2Fgrafana/155782587?tag=pr-3262

Thanks so much! I also pulled teslamate from 3262. It seems to be going offline now after letting it run overnight. https://github.com/teslamate-org/teslamate/pkgs/container/teslamate/155783116?tag=pr-3262

[USAFPride]

@micves Currently on 2023.44.30.6 and still working

[Dhtjf]

Similar on 2023.44.30.4. It's still waking every hour (1 minute long, shown as status "start") in home assistant integration linked to teslamate, but grafana doesn't show it for some reason. Also not sure the cause of the long awake times, I haven't driven in a few days and is unplugged. Lost more battery in standby compared to older Tesla versions.

[USAFPride]

@Dhtjf, the wake every hour is due to the HA integration. It is not updated with this PR for updated MCU1 -> MCU2 vehicles. If you turn off the HA integration, it will stop having the short wake cycles.

[fetzu]

@Dhtjf, the wake every hour is due to the HA integration. It is not updated with this PR for updated MCU1 -> MCU2 vehicles. If you turn off the HA integration, it will stop having the short wake cycles.

It might be a dumb question, but I didn't find anything related to "turning off the HomeAssistant integration" in the docs: how do o do that? Is not using HA enough?

[USAFPride]

@fetzu, in the HA HACS Integration, you should have a couple options:

1. Disable Polling

2. Completely disable integration

[fetzu]

@fetzu, in the HA HACS Integration, you should have a couple options:

@USAFPride Thank you for your answer, so there's nothing to turn off on TeslaMate's side if I understood correctly... which means I'm all good. Currently trying to run my setup against the :pr-3262 branch to see what happens.

[USAFPride]

@micves , is there anything we can do to get this implemented? The HA integration is based on the master and without this fixed, the integration still wakes the car constantly

[micves]

@USAFPride , the tests needs to be able to run without error. And that is a little challenging since I have changed how the internal states are running (needs info from stream api before it goes to online etc.) To be honest I haven't really prioritized it at all. Busy with work and other stuff :)

I'm also a little worried about the fact that HA see the state as start. That might mean that I need to change something in my fork to avoid that.

It is not enough to turn of polling in HA integration?

[Dhtjf]

I'll say that my HA integration shows start briefly, but the car doesn't actually start. It continues to stay asleep until the car is woke otherwise. It appears there are some API changes that will likely lead to a new release that we'll all need to use. It'd be awesome to get this merged in, if you have time to give it some work. Beggars can't be choosers and I definitely understand not having time. Either way, thank you so far!

[micves]

I have an idea and plan of changing the code to not go to start state to test for real online. But that requires some rework, and re-testing. And for now the API changes are more important ;)

I started looking a little into it today, as I got an update on the car (2023.44.30.9) and after a restart of teslamate got the same issues as in #3629

I hope I'll soon have an update with newest master and the fix on #3630

[micves]

Its updated to master and the API fixes. The images are ready if anyone dares to test :) docker pull ghcr.io/teslamate-org/teslamate:pr-3262

My car went or was online, when I started teslamate. I'll leave it on during the night and see how it goes.

[micves]

Well this is bad :(

And not a single fake online registered in the log. Seems like the hack doesn't work anymore or my car is acting up for other reasons :'(

[Dhtjf]

Think it was the car update? Something else? Are the fake online times coming through differently? My install from the prior version of the PR is still working on 2023.44.30.8 - haven't had to restart to deal with the API change.

[micves]

I'm not sure. Brought my old python script using TeslaPy back to life to test. And that seems to still work as before.

Could be a post-update hickup by the car :)

I started teslamate again with some log output and will keep an eye on it.

[micves]

No clue what was wrong, but now it seems good again.

[Dhtjf]

Great news! Thank you! Do you plan to update the PR? Apologies if you have already and I missed it. Edit: I missed it. Sorry! I'll test.

[Dhtjf]

Perfect here since updating around 24 hours ago! Thank you again! It is interesting how mine reports asleep versus offline. That started with the holiday update on my 17 S. Now on 2023.44.30.8.

[USAFPride]

@micves and @JakobLichterfeld - Do we know what is causing the testing to fail? (I know nothing about Elixir, so I don't understand it)

@micves, have you updated your fork to merge in all the changes from the master?