Home Assistant MQTT sensors unavailable due to healthy but data is available

longzheng commented 3 weeks ago

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?

I've noticed that my Home Assistant MQTT sensors sometimes report "unavailable" even though data is actually available and logged in Teslamate.

For example there was no "speed" data between 6:51am and 6:56am.

But looking at Teslamate's trip, speed was logged for the whole strip including the portion before 6:56am. I suspect this data came from the "Streaming API" which I have enabled.

Looking at some of the other sensors, I then realised that the sensor state was "unavailable" at the same time

Looking at the Home Assistant MQTT sensors documentation I then realised that all of the sensors have an availability tied to the teslamate/cars/X/healthy topic

- sensor:
    name: Speed
    object_id: tesla_speed
    unique_id: teslamate_1_speed
    availability: *teslamate_availability
    device: *teslamate_device_info
    state_topic: "teslamate/cars/1/speed"
    unit_of_measurement: "km/h"
    icon: mdi:speedometer

- sensor:
    name: Display Name
    object_id: tesla_display_name # entity_id
    unique_id: teslamate_1_display_name # internal id, used for device grouping
    availability: &teslamate_availability
      - topic: teslamate/cars/1/healthy
        payload_available: "true"
        payload_not_available: "false"
    device: &teslamate_device_info
      identifiers: [teslamate_car_1]
      configuration_url: https://teslamate.zxxz.io/
      manufacturer: Tesla
      model: Model 3
      name: Tesla Model 3
    state_topic: "teslamate/cars/1/display_name"
    icon: mdi:car

From the container logs I can verify that my vehicle became "unhealthy" around the same time due to consequtive 408 errors from the Owners API.

...
2024-10-31 06:52:38 2024-10-31 06:52:38.296 [notice]     :alarm_handler: {:set, {TeslaMate.Vehicles.Vehicle_1_api_error, :fuse_blown}}
...
2024-10-31 06:59:38 2024-10-31 06:59:38.278 [notice]     :alarm_handler: {:clear, TeslaMate.Vehicles.Vehicle_1_api_error}

I'm not familiar enough with the "healthy" state/topic, so I'm not sure if the problem is either

the healthy state should not be false if streaming data is available (as it was in my scenario)
the Home Assistant MQTT sensors should not use healthy topic as an availabillity source since it may be misleading

Expected Behavior

Home Assistant sensors to have data if data is available

Steps To Reproduce

No response

Relevant log output

2024-10-31 06:49:45 2024-10-31 06:49:45.568 car_id=1 [info] Start / :online
2024-10-31 06:49:45 2024-10-31 06:49:45.578 car_id=1 [info] Connecting ...
2024-10-31 06:50:24 2024-10-31 06:50:24.998 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/XXXX/vehicle_data -> 408 (9420.470 ms)
2024-10-31 06:50:24 2024-10-31 06:50:24.998 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-10-31 06:50:24 2024-10-31 06:50:24.998 car_id=1 [error] Error / :unknown
2024-10-31 06:50:54 2024-10-31 06:50:54.301 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/XXXX/vehicle_data -> 408 (9302.856 ms)
2024-10-31 06:50:54 2024-10-31 06:50:54.301 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-10-31 06:50:54 2024-10-31 06:50:54.301 car_id=1 [error] Error / :unknown
2024-10-31 06:51:23 2024-10-31 06:51:23.614 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/XXXX/vehicle_data -> 408 (9313.156 ms)
2024-10-31 06:51:23 2024-10-31 06:51:23.614 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-10-31 06:51:23 2024-10-31 06:51:23.615 car_id=1 [error] Error / :unknown
2024-10-31 06:51:52 2024-10-31 06:51:52.911 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/XXXX/vehicle_data -> 408 (9297.670 ms)
2024-10-31 06:51:52 2024-10-31 06:51:52.911 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-10-31 06:51:52 2024-10-31 06:51:52.912 car_id=1 [error] Error / :unknown
2024-10-31 06:52:22 2024-10-31 06:52:22.307 [info] GET https://owner-api.teslamotors.com/api/1/vehicles/XXXX/vehicle_data -> 408 (9394.947 ms)
2024-10-31 06:52:22 2024-10-31 06:52:22.307 [warning] TeslaApi.Error / %{"error" => "{\"error\": \"timeout\"}", "error_description" => "", "response" => nil}
2024-10-31 06:52:22 2024-10-31 06:52:22.307 car_id=1 [error] Error / :unknown
2024-10-31 06:52:38 2024-10-31 06:52:38.296 [notice]     :alarm_handler: {:set, {TeslaMate.Vehicles.Vehicle_1_api_error, :fuse_blown}}
2024-10-31 06:53:30 2024-10-31 06:53:30.082 [info] Start of drive initiated by: %TeslaApi.Vehicle.State.Drive{active_route_destination: "Work", active_route_energy_at_arrival: 75, active_route_latitude: XXXX, active_route_longitude: XXXX, active_route_miles_to_arrival: 11.42448, active_route_minutes_to_arrival: 29.183333, active_route_traffic_minutes_delay: 0.0, gps_as_of: 1730318009, heading: 190, latitude: XXXX, longitude: XXXX, native_latitude: XXXX, native_location_supported: 1, native_longitude: XXXX, native_type: "wgs", power: 3, shift_state: "D", speed: 1, timestamp: 1730318009947}
2024-10-31 06:53:30 2024-10-31 06:53:30.082 car_id=1 [info] Driving / Start
2024-10-31 06:59:38 2024-10-31 06:59:38.278 [notice]     :alarm_handler: {:clear, TeslaMate.Vehicles.Vehicle_1_api_error}
2024-10-31 07:25:46 2024-10-31 07:25:46.976 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (964.868 ms)
2024-10-31 07:25:47 2024-10-31 07:25:47.492 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (514.516 ms)
2024-10-31 07:25:47 2024-10-31 07:25:47.501 [info] End of drive initiated by: %TeslaApi.Vehicle.State.Drive{active_route_destination: nil, active_route_energy_at_arrival: 75, active_route_latitude: XXXX, active_route_longitude: XXXX, active_route_miles_to_arrival: nil, active_route_minutes_to_arrival: nil, active_route_traffic_minutes_delay: 0.0, gps_as_of: 1730319944, heading: 101, latitude: XXXX, longitude: XXXX, native_latitude: XXXX, native_location_supported: 1, native_longitude: XXXX, native_type: "wgs", power: 2, shift_state: "P", speed: 0, timestamp: 1730319945904}
2024-10-31 07:25:47 2024-10-31 07:25:47.501 car_id=1 [info] Driving / Ended / 19 km – 32 min

Screenshots

No response

Additional data

No response

Type of installation

Docker

Version

v1.31.1

JakobLichterfeld commented 3 weeks ago

Thanks for reporting, nice catch.

the healthy state should not be false if streaming data is available (as it was in my scenario)

the Home Assistant MQTT sensors should not use healthy topic as an availabillity source since it may be misleading

I do agree to both

longzheng commented 3 weeks ago

Is there an alternative to "healthy" that can be used for the MQTT sensor availability?

brianmay commented 3 weeks ago

What do we want "healthy" to indicate?

Maybe a car that drives into and out of a mobile black spot should still be considered healthy?

What if the car drives into a black spot and parks there overnight?

Could be tricky. How do we distinguish a problem that should be ignored vs one that should not be ignored?

longzheng commented 3 weeks ago

By no means I'm a Home Assistant expert, but I do wonder if the sensor "availability" is sort of misleading here since there's no correlation between sensor values/states like "location" and "speed" with "healthy" or "not healthy".

The way I'm thinking about it, the "location" or "speed" sensor only updates if there is a new value pushed from MQTT. If there is no value, the last value/state is persisted in Home Assistant with a timestamp of when it was set. So if the car is offline/unreachable, then the MQTT topic will not have any new values and the sensor will not update, then the last value is the last known value for that sensor.

I believe that's also what Teslamate's own dashboard would show as well, the last known location and speed if the car suddenly became unreachable. There is a "car is unhealthy" indicator but it doesn't hide/stop all the other values from being shown.

Since there is in fact a "Healthy" sensor tied to the healthy topic, if anyone/anything was concerned with the current "health" of the connection, then they can use that sensor value directly.

However for certain sensors like the "Active route", here the "availability" topic is meaningful because if the car is not actively routing, then the "Active route" should not be the last set state because that's not reflective of the actual state, so the "Unavailable" state has meaning here.

longzheng commented 2 weeks ago

@t3hk0d3 I saw that you added the sensor availability attributes in https://github.com/teslamate-org/teslamate/pull/3344

Do you have any opinions or suggestions as to whether they can be removed?

longzheng commented 2 weeks ago

@JakobLichterfeld since I haven't heard back from anyone about this, my opinion is that the availability attribute should removed from most sensors (excluding the active route ones). I'll open a PR for this.

JakobLichterfeld commented 2 weeks ago

@JakobLichterfeld since I haven't heard back from anyone about this, my opinion is that the availability attribute should removed from most sensors (excluding the active route ones).

Agree.

I'll open a PR for this.

Ty!

teslamate-org / teslamate