MQTT discovery items end up in 'unknown' state after HA restart

[MichielVanwelsenaere]

The problem

After restarting Home Assistant a arbitrary number of MQTT entities in my device end up in an 'unknown' state. All device entities are configured via discovery:

When doublechecking the broker the state messages representing the current state of the light, switch etc are present with a retain flag:

When I restart HA again, the issue persist but often on other entities then before the restart. I've attached the MQTT diagnostic logs, let me know in case I can provide any more information.

What version of Home Assistant Core has the issue?

2023.2.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

mqtt

Link to integration documentation on our website

https://www.home-assistant.io/integrations/mqtt/

Diagnostics information

mqtt-b70478debc6d480585f67375ab7db08c-PLC_Woning-9eaee25ab4ade0b036628d7606eb6eb7.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @emontnemery, @jbouwh, mind taking a look at this issue as it has been labeled with an integration (mqtt) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `mqtt` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign mqtt` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

mqtt documentation mqtt source _{^{(message by IssueLinks)}}

[jbouwh]

This is as designed and not an issue. The state restores after a new state is received. If the state is published as a retained message the state will restore after restarting (or reconnecting)

[jbouwh]

Sorry to fast judgement. Can provide steps that allows us to reproduce your issue?

[MichielVanwelsenaere]

Hi Jan,

no problem. If my understanding of the mqtt discovery principles is faulty in anyway I'd like to know so I can adjust my implementation.

Are the steps below sufficient in terms of reproducing the issue?

1. send mqtt discovery message to broker with retain flag (for example for a switch entity)
2. send mqtt state message for the switch entity with retain flag
3. entity should be provisioned succesfully in HA
4. restart HA
5. entity has a 'unknown' state

as mentioned: it's quite arbitrary, if I restart HA I've got some entities who suddenly get an 'unkown' status and other ones that provision just fine. If I restart again some of the entities with an 'unkown' status get provisioned fine and other which were fine before the restart get the 'unkown' issue

[jbouwh]

What can happen is that the retained state message is replayed before the config message, when that happens the state will be unknown.

[MichielVanwelsenaere]

you mean on startup of HA when all messages from the broker are read? Because both the config message and state message are already on the broker before the startup is triggered.

Is this to be considered a bug on HA end or something that systems leveraging mqtt discovery should coop with somehow?

[jbouwh]

Is this to be considered a bug on HA end or something that systems leveraging mqtt discovery should coop with somehow?

Config payloads can be retained (they are catched though the discovery wildcard topic). Other topics are subscribed when the configs are processed. When subscribing any retained messages should be retained as soon as the subscriptions takes place. A current bug is that when a topic (e.g. a state or availability topic) is shared, and the state is retained, the retained payload is replayed for the new subscriptions. In the current code the side effect is that also existing subscriptions are replayed. That could cause issues. https://github.com/home-assistant/core/pull/88826 is addressing this and https://github.com/home-assistant/core/pull/88862 optimizes subscribing to shared topics to be grouped and delayed. But I will test with your suggested steps to see if this is something that is expected.

[jbouwh]

I have been testing with 20 retained entities (without the mentioned PR's) but I cannot reproduce what you are seeing. Not sure what is happening. How many entities are set up this way?

[MichielVanwelsenaere]

Investigating this in more depth after your feedback I updated my mqtt broker to the latest version and I can no longer reproduce the issue. My apologies for wasting your time.

I have a small side question I'd like to get your opinion on. For the last years I've been developing an open source PLC home automation project (link) and lately MQTT discovery support has been added. One of the challenges that I'm facing (due to the IEC 61131-3 standard that is involved in PLC programming) is that optional arguments like 'icon' and 'device_class' have a 'null' value if the user doesn't provide a value. Unfortunately the mqtt discovery logic breaks on this. Do you see it feasible to just ignore 'null' properties in the mqtt discovery logic? It's a bit debatable whether this is to be considered a bug or a feature request so not sure if I need to open another git issue or go to the forum for this.

Thanks for the support, much appreciated!

[jbouwh]

Thnx for reporting, you did not waist my time btw !