search

arendst / Tasmota

Alternative firmware for ESP8266 and ESP32 based devices with easy configuration using webUI, OTA updates, automation using timers or rules, expandability and entirely local control over MQTT, HTTP, Serial or KNX. Full documentation at
https://tasmota.github.io/docs
GNU General Public License v3.0
21.97k stars 4.77k forks source link

MQTT Reconnect does not send Home Assistant DISCOVERY topic leading to Unavailable device. #12140

Closed CRCinAU closed 3 years ago

CRCinAU commented 3 years ago

PROBLEM DESCRIPTION

When Tasmota reboots, it will send the DISCOVERY topic to the MQTT server - which HomeAssistant picks up and makes the device available.

If you reboot HA / MQTT, the DISCOVERY topic is gone and therefore HA marks the device unavailable. When Tasmota reconnects to the MQTT server, it does not send the DISCOVERY topic to the MQTT server.

As such, the device will always show as unavailable in Home Assistant until the DISCOVERY topic is published again. Currently, this will only happen on rebooting the Tasmota device.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

- [ ] If using rules, provide the output of this command: `Backlog Rule1; Rule2; Rule3`:
```lua
  Rules output here:
N/A

- [ ] Set `weblog` to 4 and then, when you experience your issue, provide the output of the Console log:
```lua
  Console output here:
N/A

TO REPRODUCE

Reboot HA / MQTT system losing all current topics.

EXPECTED BEHAVIOUR

DISCOVERY topic published on reconnect to MQTT.

SCREENSHOTS

N/A

ADDITIONAL CONTEXT

N/A

ascillato commented 3 years ago

Hi,

The discovery message is retained, so your issue is not that unless you have some custom settings in your broker that don't allow retained messages?

Home Assistant see a device as available or unavailable due to the LWT message, not because of the discovery message.

I couldn't reproduce your issue with Standard Tasmota, Standard MQTT and Home Assistant.

Seems that your issue is elsewhere, not in Tasmota, sorry.

Please, share the logs of your MQTT broker, Tasmota device and HA.

CRCinAU commented 3 years ago

Ok, so on boot, I see the following messages from the Tasmota device:

tele/tasmota_E0B7C9/LWT Offline
cmnd/tasmota_E0B7C9/STATE (null)
cmnd/tasmota_E0B7C9/STATUS 10
** reboot **
tele/tasmota_E0B7C9/LWT Online
cmnd/tasmota_E0B7C9/POWER (null)

tele/tasmota_E0B7C9/INFO1 {"Info1":{"Module":"Sonoff Pow R2","Version":"9.4.0(tasmota)","FallbackTopic":"cmnd/DVES_E0B7C9_fb/","GroupTopic":"cmnd/tasmotas/"}}
tele/tasmota_E0B7C9/INFO2 {"Info2":{"WebServerMode":"Admin","Hostname":"tasmota_E0B7C9-6089","IPAddress":"172.31.1.110"}}
tele/tasmota_E0B7C9/INFO3 {"Info3":{"RestartReason":"Software/System restart"}}
stat/tasmota_E0B7C9/RESULT {"POWER":"ON"}
stat/tasmota_E0B7C9/POWER ON
cmnd/tasmota_E0B7C9/STATE (null)
cmnd/tasmota_E0B7C9/STATUS 10
stat/tasmota_E0B7C9/RESULT {"Time":"2021-05-20T22:08:21","Uptime":"0T00:00:05","UptimeSec":5,"Heap":28,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"POWER":"ON","Wifi":{"AP":1,"SSId":"<SSID>","BSSId":"40:01:7A:38:32:21","Channel":1,"RSSI":100,"Signal":-40,"LinkCount":1,"Downtime":"0T00:00:03"}}
stat/tasmota_E0B7C9/STATUS10 {"StatusSNS":{"Time":"2021-05-20T22:08:21","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.666,"Yesterday":5.236,"Today":4.592,"Power":246,"ApparentPower":275,"ReactivePower":123,"Factor":0.89,"Voltage":235,"Current":1.168}}}
tele/tasmota_E0B7C9/STATE {"Time":"2021-05-20T22:08:25","Uptime":"0T00:00:09","UptimeSec":9,"Heap":28,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"POWER":"ON","Wifi":{"AP":1,"SSId":"<SSID>","BSSId":"40:01:7A:38:32:21","Channel":1,"RSSI":100,"Signal":-39,"LinkCount":1,"Downtime":"0T00:00:03"}}
tele/tasmota_E0B7C9/SENSOR {"Time":"2021-05-20T22:08:25","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.667,"Yesterday":5.236,"Today":4.593,"Period":0,"Power":252,"ApparentPower":287,"ReactivePower":137,"Factor":0.88,"Voltage":235,"Current":1.223}}

If I reboot the VM hosting Home Assistant, when Tasmota connects, and do not reboot the tasmota device, I see:

tele/tasmota_E0B7C9/LWT Online
tele/tasmota_E0B7C9/STATE {"Time":"2021-05-20T22:11:55","Uptime":"0T00:03:39","UptimeSec":219,"Heap":26,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":5,"POWER":"ON","Wifi":{"AP":1,"SSId":"<SSID>","BSSId":"40:01:7A:38:32:21","Channel":1,"RSSI":100,"Signal":-42,"LinkCount":1,"Downtime":"0T00:00:03"}}
tele/tasmota_E0B7C9/SENSOR {"Time":"2021-05-20T22:11:55","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.682,"Yesterday":5.236,"Today":4.608,"Period":2,"Power":253,"ApparentPower":276,"ReactivePower":110,"Factor":0.92,"Voltage":234,"Current":1.179}}

After the full reboot, there is no retained DISCOVERY or other topics that are published when Tasmota reboots & connects for the first time.

I only see a single retained topic after this:

tele/tasmota_E0B7C9/LWT Online

I'm monitoring all topics via the command mosquitto_sub -v -t '#'

CRCinAU commented 3 years ago

At this point, Home Assistant shows: image image

CRCinAU commented 3 years ago

Rebooting Tasmota again, I see the following:

tele/tasmota_E0B7C9/LWT Online
cmnd/tasmota_E0B7C9/POWER (null)
tele/tasmota_E0B7C9/INFO1 {"Info1":{"Module":"Sonoff Pow R2","Version":"9.4.0(tasmota)","FallbackTopic":"cmnd/DVES_E0B7C9_fb/","GroupTopic":"cmnd/tasmotas/"}}
tele/tasmota_E0B7C9/INFO2 {"Info2":{"WebServerMode":"Admin","Hostname":"tasmota_E0B7C9-6089","IPAddress":"172.31.1.110"}}
tele/tasmota_E0B7C9/INFO3 {"Info3":{"RestartReason":"Software/System restart"}}
stat/tasmota_E0B7C9/RESULT {"POWER":"ON"}
stat/tasmota_E0B7C9/POWER ON
tele/tasmota_E0B7C9/STATE {"Time":"2021-05-20T22:24:55","Uptime":"0T00:00:09","UptimeSec":9,"Heap":28,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"POWER":"ON","Wifi":{"AP":1,"SSId":"<SSID>","BSSId":"40:01:7A:38:32:21","Channel":1,"RSSI":100,"Signal":-40,"LinkCount":1,"Downtime":"0T00:00:03"}}
tele/tasmota_E0B7C9/SENSOR {"Time":"2021-05-20T22:24:55","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.737,"Yesterday":5.236,"Today":4.663,"Period":0,"Power":256,"ApparentPower":275,"ReactivePower":102,"Factor":0.93,"Voltage":232,"Current":1.185}}
tasmota/discovery/98F4ABE0B7C9/config {"ip":"172.31.1.110","dn":"UPS","fn":["UPS Switch",null,null,null,null,null,null,null],"hn":"tasmota_E0B7C9-6089","mac":"98F4ABE0B7C9","md":"Sonoff Pow R2","ty":0,"if":0,"ofln":"Offline","onln":"Online","state":["OFF","ON","TOGGLE","HOLD"],"sw":"9.4.0","t":"tasmota_E0B7C9","ft":"%prefix%/%topic%/","tp":["cmnd","stat","tele"],"rl":[1,0,0,0,0,0,0,0],"swc":[-1,-1,-1,-1,-1,-1,-1,-1],"swn":[null,null,null,null,null,null,null,null],"btn":[0,0,0,0,0,0,0,0],"so":{"4":0,"11":0,"13":0,"17":0,"20":0,"30":0,"68":0,"73":0,"82":0,"114":0,"117":0},"lk":1,"lt_st":0,"sho":[0,0,0,0],"ver":1}
tasmota/discovery/98F4ABE0B7C9/sensors {"sn":{"Time":"2021-05-20T22:25:00","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.737,"Yesterday":5.236,"Today":4.663,"Power":265,"ApparentPower":276,"ReactivePower":78,"Factor":0.96,"Voltage":233,"Current":1.184}},"ver":1}

cmnd/tasmota_E0B7C9/STATE (null)
stat/tasmota_E0B7C9/RESULT {"Time":"2021-05-20T22:25:01","Uptime":"0T00:00:15","UptimeSec":15,"Heap":26,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"POWER":"ON","Wifi":{"AP":1,"SSId":"<SSID>","BSSId":"40:01:7A:38:32:21","Channel":1,"RSSI":100,"Signal":-40,"LinkCount":1,"Downtime":"0T00:00:03"}}
cmnd/tasmota_E0B7C9/STATUS 10
stat/tasmota_E0B7C9/STATUS10 {"StatusSNS":{"Time":"2021-05-20T22:25:01","ENERGY":{"TotalStartTime":"2021-03-24T13:50:10","Total":274.737,"Yesterday":5.236,"Today":4.663,"Power":254,"ApparentPower":278,"ReactivePower":113,"Factor":0.91,"Voltage":233,"Current":1.192}}}

At this point, Home Assistant changes to: image

rapi3 commented 3 years ago

When MQTT broker or Home Assistant is restarted, or there is a WiFi outage, Tasmota device states may not be synced with Home Assistant. Use this automation to keep your devices in sync, including power state, immediately after Home Assistant is started. https://tasmota.github.io/docs/Home-Assistant/#useful-automations

ascillato commented 3 years ago

Hi,

Please, recheck your mqtt broker.

This issue is duplicated from

https://github.com/arendst/Tasmota/discussions/12142

And

https://github.com/arendst/Tasmota/issues/8753

CRCinAU commented 3 years ago

Ok, so to give this a reasonable fix, I turned on storage of persistent messages to disk in mosquitto.

It feels like a dirty fix, and I'm not sure why we seem to be so against sending it on reconnect - given that its trivial and fixes so many corner cases - but given that I guess that's the decision, that will have to do...

It feels cleaner to start an MQTT server with nothing stored, and then things publish their own data if its missing - and certainly that's how I've always coded my MQTT operations - but if that's not what is wanted, then I guess that's where we are.

ascillato2 commented 3 years ago

Great to know that you could solve your issue.

CRCinAU commented 3 years ago

Great to know that you could solve your issue.

It's not really a solution - but a workaround. It has multiple draw backs...

But I guess based on the other threads that you referenced, this is not considered a viable solution - leaving only scripting hacks or workarounds to fix the core issue.

rapi3 commented 3 years ago

why not make your rule and do exactly what you want/need ... just copy/paste the original Tasmota discovery topic and you are done. ON mqtt#connected DO Publish2 homeassistant/....discovery topic... ENDON

CRCinAU commented 3 years ago

why not make your rule and do exactly what you want/need

It's still a hack to fix a core problem....

However I just realised that if you're going to go through the effort to subscribe to a topic to make sure your discovery data is up to date, then you may as well just publish it again anyway - its less time and resources - and increases reliability...

I guess I see it more from the position of which is easier - a fix in the code, or 1000 users doing workarounds... But then its not my codebase hahah :)

rapi3 commented 3 years ago

What you need it is not always what others need so it is best to fork the code and change it as you need or do your hacks.

CRCinAU commented 3 years ago

What you need it is not always what others need

I guess you're right - but I can't figure out a usage where someone would say "I wish that my home assistant integration broke every time I rebooted my Home Assistant system" ;)

But that's fine - I'll leave it be - even though I don't understand the logic to not fix this....

rapi3 commented 3 years ago

From my one year HA experience: Home Assistant is the most unreliable piece of sw I ever meet in my life until now... it looks like a continuous alpha code work patched/hack. Tried first on rpi - it is a long pain to reboot, almost every update broke something that worked before... totally unreliable. Finally I moved HA to a virtual box ( much easy to take snapshots and restore after a failed upgrade and quick reboot ) and I cut access for HA to internet because it does not respect local DNS server and it is to chatty with home server on internet; moved mqtt server to linux + another mqtt bridge as backup and slowly try to make my critical home automation only with Tasmota & rules and without HA involvement. Thanks to arendst & all developers for Tasmota.

joba-1 commented 3 years ago

But that's fine - I'll leave it be - even though I don't understand the logic to not fix this....

maybe you are barking at the wrong tree?

Tasmota sends a message when it boots, it sends messages on status change and even tele messages at regular intervals. HA has every chance to pick them up, discover or rediscover devices and even query their status if it lost them.

Then it is not Tasmotas fault if HA restarts nor can it know that this happened while HA obviously can,

Finally: Tasmota devices are low on resources compared to a server, even a Pi, so additional burdens should be on the server side. Tasmota can not and should not need to adapt for special cases of every home automation system there is.

CRCinAU commented 3 years ago

Then it is not Tasmotas fault if HA restarts nor can it know that this happened while HA obviously can,

I feel you misunderstand the problem with this simplistic analysis....

Finally: Tasmota devices are low on resources compared to a server, even a Pi, so additional burdens should be on the server side. Tasmota can not and should not need to adapt for special cases of every home automation system there is.

I know very well what this stuff can and can't do - I helped debug the Arduino framework for the ESP8266 to fix sleep issues. I'm not a stranger to this environment.

Anyway - I think its a call to HAssDiscovery() inside an #ifdef USE_HOME_ASSISTANT in MqttReconnect() and that's about all that's required. At least from a 10 minute review of the existing code...

ascillato commented 3 years ago

Anyway - I think its a call to HAssDiscovery() inside an #ifdef USE_HOME_ASSISTANT in MqttReconnect() and that's about all that's required.

Let's ask @emontnemery what he thinks about this.

emontnemery commented 3 years ago

Yes, I agree it's reasonable to resend the discovery message

arendst commented 3 years ago

I'll add it today

CRCinAU commented 3 years ago

Thanks guys - I'm sure this will fix a lot of integrations and not require workarounds.

Would this likely end up in release 9.4.1 or later? I'm not quite sure how you guys manage the release cycles.

arendst commented 3 years ago

See https://github.com/arendst/Tasmota/milestone/16

In the meantime just use the daily compiled dev binaries

CRCinAU commented 3 years ago

In the meantime just use the daily compiled dev binaries

Ah cool - that's a nice CI setup... I gave the dev version a spin, powered off the HA/MQTT VM, let it fail to connect for a while, then powered the VM back on again and waited.

I can confirm that the discovery topic is published again, and the sensor springs back into life in HA without any further manual intervention or script work.

Thanks again.

digiblur commented 3 years ago

Ok, so to give this a reasonable fix, I turned on storage of persistent messages to disk in mosquitto.

It feels like a dirty fix, and I'm not sure why we seem to be so against sending it on reconnect - given that its trivial and fixes so many corner cases - but given that I guess that's the decision, that will have to do...

It feels cleaner to start an MQTT server with nothing stored, and then things publish their own data if its missing - and certainly that's how I've always coded my MQTT operations - but if that's not what is wanted, then I guess that's where we are.

Without persistence turned on you are totally killing the way MQTT is used for discovery devices. Just write it to disk every 5-10 minutes or something, not many writes there especially if there are no changes in retains. Then rely on LWT like it was designed.