Home Assistant can't decode payload

GVollaro commented 3 years ago

PROBLEM DESCRIPTION

A clear and concise description of what the problem is. I made a weather station with ESP8266 equipped with BME280, pulse anemometer, analog wind direction sensor and sw Tasmota. The data is used by Home Assistant and everything worked perfectly until some time ago, probably until I updated the firmware with version 9. From that moment on, the analog wind direction sensor is not recognized by Home Assistant; this problem also occurred when switching to 9.2 and 9.3. The value is however correctly reported on the Tasmota WebGui. Analyzing the HA log I found the following message:

Logger: homeassistant.components.mqtt Source: components/mqtt/init.py:879 Integration: MQTT (documentation, issues) First occurred: 18:11:30 (1 occurrences) Last logged: 18:11:30

Can't decode payload b'{"name":"SensorEXT ANALOG A0","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"1\xea\x02\x002homeassistant/sensor/589755_BME280_Pressure/config{"name":"SensorEXT BME280 Pressure","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"Online","pl_n' on homeassistant/sensor/589755_ANALOG_A0/config with encoding utf-8 (for <Job HassJobType.Coroutinefunction <function async_start..async_discovery_message_received at 0xae56f1d8>>)

which shows that the config message "homeassistant / sensor / 589755_ANALOG_A0 / config" is not decoded. Going to read the related topic through MQTT Explorer, I found the following value:

{"name":"SensorEXT ANALOG A0","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"1�2homeassistant/sensor/589755_BME280_Pressure/config{"name":"SensorEXT BME280 Pressure","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"Online","pl_n

which is clearly altered. The topics "* / config" related to all the other sensors of the device (Counter1_01, BME280_Temperature, BME280_Humidity, BME280_Pressure, BME280_SeaLevelPreassure) are correct. I tried both to publish the config with the correct format, which is then covered by the wrong one, and to insert the correct configuration directly in the file "/config/.storage/core.entity_registry" without results. At this point I think the source of the problem is in an incorrect formatting of the config message sent by the Tasmota due to a firmware bug. Thanks for your support Regards

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

[x ] Read the Contributing Guide and Policy and the Code of Conduct
[x ] Searched the problem in issues
[x ] Searched the problem in discussions
[x ] Searched the problem in the docs
[x ] Searched the problem in the chat
[ ] Device used (e.g., Sonoff Basic): __NodeMCU esp8266___
[ ] Tasmota binary firmware version number used: __9.3.1___
- [x ] Pre-compiled
- [x ] Self-compiled
[x ] Flashing tools used: __OTA___

[x ] Provide the output of command: Backlog Template; Module; GPIO 255:

Configuration output here:
SensorEXT/stat/RESULT = {"NAME":"Meteo","GPIO":[1,1,1,1,640,608,1,1,160,1,352,1,1,4704],"FLAG":0,"BASE":18}
SensorEXT/stat/RESULT = {"Module":{"0":"Meteo"}}
SensorEXT/stat/RESULT = {"GPIO0":{"0":"Nessuno"},"GPIO1":{"0":"Nessuno"},"GPIO2":{"0":"Nessuno"},"GPIO3":{"0":"Nessuno"},"GPIO4":{"640":"I2C SDA"},"GPIO5":{"608":"I2C SCL"},"GPIO9":{"0":"Nessuno"},"GPIO10":{"0":"Nessuno"},"GPIO12":{"160":"Switch1"},"GPIO13":{"0":"Nessuno"},"GPIO14":{"352":"Contatore1"},"GPIO15":{"0":"Nessuno"},"GPIO16":{"0":"Nessuno"},"GPIO17":{"4704":"ADC - Ingresso"}}

[ ] If using rules, provide the output of this command: Backlog Rule1; Rule2; Rule3:

Rules output here:
21:21:35.992 MQT: SensorEXT/stat/RESULT = {"Rule1":"ON","Once":"OFF","StopOnError":"OFF","Length":40,"Free":471,"Rules":"on tele-counter#c1>0 do counter1 0 endon"}
21:21:36.196 MQT: SensorEXT/stat/RESULT = {"Rule2":"OFF","Once":"OFF","StopOnError":"OFF","Length":0,"Free":511,"Rules":""}
21:21:36.399 MQT: SensorEXT/stat/RESULT = {"Rule3":"OFF","Once":"OFF","StopOnError":"OFF","Length":0,"Free":511,"Rules":""}

[ ] Provide the output of this command: Status 0:


STATUS 0 output here:
SensorEXT/stat/STATUS = {"Status":{"Module":0,"DeviceName":"SensorEXT","FriendlyName":["SensorEXT"],"Topic":"SensorEXT","ButtonTopic":"SensorEXT","Power":0,"PowerOnState":3,"LedState":1,"LedMask":"FFFF","SaveData":1,"SaveState":1,"SwitchTopic":"SensorEXT","SwitchMode":[2,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0,"InfoRetain":0,"StateRetain":0}}

- [ ] Set `weblog` to 4 and then, when you experience your issue, provide the output of the Console log:
```lua
  Console output here:

TO REPRODUCE

Steps to reproduce the behavior: An ESP8266 with the reported configuration

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen.

SCREENSHOTS

If applicable, add screenshots to help explain your problem.

ADDITIONAL CONTEXT

Add any other context about the problem here. Home Assistant Log:

Logger: homeassistant.components.mqtt Source: components/mqtt/init.py:879 Integration: MQTT (documentation, issues) First occurred: 18:11:30 (1 occurrences) Last logged: 18:11:30

Can't decode payload b'{"name":"SensorEXT ANALOG A0","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"1\xea\x02\x002homeassistant/sensor/589755_BME280_Pressure/config{"name":"SensorEXT BME280 Pressure","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"Online","pl_n' on homeassistant/sensor/589755_ANALOG_A0/config with encoding utf-8 (for <Job HassJobType.Coroutinefunction <function async_start..async_discovery_message_received at 0xae56f1d8>>)

MQTT message (as reported in MQTT Explorer): homeassistant / sensor / 589755_ANALOG_A0 / config = {"name":"SensorEXT ANALOG A0","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"1�2homeassistant/sensor/589755_BME280_Pressure/config{"name":"SensorEXT BME280 Pressure","stat_t":"SensorEXT/tele/SENSOR","avty_t":"SensorEXT/tele/LWT","pl_avail":"Online","pl_n

(Please, remember to close the issue when the problem has been addressed)

sfromis commented 3 years ago

To me, the variety of broken data looks like severe problems with settings. What version were you running before upgrading to 9? If you did not follow the stipulated upgrade flow, that may be the cause. In that case, I'd recommend resetting to "factory defaults" and reconfigure from scratch.

GVollaro commented 3 years ago

Before version 9 I had installed 8.5. I also installed the firmware with tasmotizer and "erase before flashing" option

sfromis commented 3 years ago

Well, due to the signs of settings data breakage, I'd still suggest the "factory reset" and reconfigure from scratch. It the issue persists, please post configuration details (what you changed going from scratch).

GVollaro commented 3 years ago

I just cleaned Home Assistant and Mosquitto Brocker of any traces of the device, I reinstalled the firmware 9.3.1 with OTA keeping the current configuration and finally I waited for Home Assistant to discover the device again; now everything works smoothly. Probably some old traces of the device prevented an update of the corrupted topic "homeassistant / sensor / 589755_COUNTER_C1". Anyway, thanks to @Sfromis for his suggestions.

pickonedev commented 1 year ago

I have the same problem, I have installed a newer firmware (13.0.0), factory reset, everything, I still get errors about decoding payload and utf-8 thing... Then, I bought a new Zigbee Sonoff Bridge, tasmotized, all from scratch, same error... Something what my zigbee sensors are doing, tasmota don't like it... I don't know

This is from Home Assistant logs...

Logger: homeassistant.components.mqtt.client
Source: components/mqtt/client.py:823
Integration: MQTT ([documentation](https://www.home-assistant.io/integrations/mqtt), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+mqtt%22))
First occurred: 18:37:29 (144 occurrences)
Last logged: 18:51:15

    Can't decode payload b'{"ZbReceived":{"Living_Ambient":{"Device":"0xC877","Name":"Living_Ambient","0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i","Endpoint":1,"LinkQuality":58}}}' on tele/ZigBee_Bridge/C877/SENSOR with encoding utf-8 (for <Job None HassJobType.Callback <function async_attach_trigger.<locals>.mqtt_automation_listener at 0x7fe446d765c0>>)
    Can't decode payload b'{"ZbReceived":{"Living_Ambient":{"Device":"0xC877","Name":"Living_Ambient","0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i","Endpoint":1,"LinkQuality":58}}}' on tele/ZigBee_Bridge/C877/SENSOR with encoding utf-8 (for <Job None HassJobType.Callback <function async_attach_trigger.<locals>.mqtt_automation_listener at 0x7fe446d76b60>>)
    Can't decode payload b'{"ZbReceived":{"Living_Ambient":{"Device":"0xC877","Name":"Living_Ambient","0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i","Endpoint":1,"LinkQuality":58}}}' on tele/ZigBee_Bridge/C877/SENSOR with encoding utf-8 (for <Job None HassJobType.Callback <function async_attach_trigger.<locals>.mqtt_automation_listener at 0x7fe446d77100>>)
    Can't decode payload b'{"ZbReceived":{"Living_Ambient":{"Device":"0xC877","Name":"Living_Ambient","0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i","Endpoint":1,"LinkQuality":58}}}' on tele/ZigBee_Bridge/C877/SENSOR with encoding utf-8 (for <Job None HassJobType.Callback <function async_attach_trigger.<locals>.mqtt_automation_listener at 0x7fe446d776a0>>)
    Can't decode payload b'{"ZbReceived":{"Living_Ambient":{"Device":"0xC877","Name":"Living_Ambient","0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i","Endpoint":1,"LinkQuality":58}}}' on tele/ZigBee_Bridge/C877/SENSOR with encoding utf-8 (for <Job None HassJobType.Callback <function async_attach_trigger.<locals>.mqtt_automation_listener at 0x7fe446d660c0>>)

I bet this line is the cause "0000/FFDF":"\xa3R@,i\xa3R@,i\xa8R@,i\xa8R@,i\xa8R@,i\xa9R@,i\xa9R@,i\xaaR@,i", but how can I solve the problem? I don't have only one sensor like this, I have 15... I get the same problem for every device...

sfromis commented 1 year ago

As your issue is Zigbee-related, it has nothing to do with this old thread. Posting in such an outdated thread, with only superficial symptom similarities cannot be recommended.

In general, to see what happens in Tasmota, look in Tasmota logs, not in HA logs. Plausibly, that 0000/FFDF cluster/attribute may need some decoding, it that payload is actually more than just "noise". It does seem to be associated with binary data, which may break UTF-8 decoding, but that does not mean much if the payload is not "needed".

It "happens" that some vendors are pretty "lax" about adhering to standards, and make their own formats needing special decoding. Especially if Tuya is involved..... My knowledge does not include specific suggestions.

pickonedev commented 1 year ago

Yes, you are right, it is a tuya device... :-( The one with 6 sensors (air quality...) The strange thing is that till yesterday, everything was good.... :-(

arendst / Tasmota