Entity does not exist

[sharondagan]

I've replaced my one of my nodes with other RPi that reports the same location etc, now confidence levels aren't updated. Also, I can see several Entity sensor.monitor_xxxxxx_conf does not exist in the logs

[Odianosen25]

@sharondagan thats interesting. Can I know what version you on, and kindly let me see the logs please?

Also will not be a bad idea to run the app in DEBUG mode for a while, so we could better figure out what’s going on.

Thanks

[sharondagan]

Yes. Version is 2.3.0.

Here's a sample from the logs, I hope it'll help.

2020-03-11 09:58:54.320601 DEBUG home_presence_app: Recieved an RSSI of -99 for sharon from Upstairs
2020-03-11 09:58:54.321682 DEBUG home_presence_app: set state: monitor.sharon_upstairs, {'attributes': {'rssi': '-99', 'last_reported_by': 'Upstairs'}}
2020-03-11 09:58:54.325808 ERROR home_presence_app: Entity sensor.monitor_sharon_upstairs_conf does not exist.
2020-03-11 09:58:54.329013 DEBUG home_presence_app: update_hass_sensor: Entity_ID: binary_sensor.monitor_sharon, new_state: off
2020-03-11 09:58:54.329773 DEBUG home_presence_app: set state: binary_sensor.monitor_sharon, {'state': 'off', 'attributes': {'device_class': 'presence', 'friendly_name': 'Sharon Home', 'nearest_monitor': 'Unknown', 'type': 'KNOWN_MAC'}}
2020-03-11 09:59:00.009506 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/echo', 'payload': ''}
2020-03-11 09:59:00.374063 DEBUG home_presence_app: monitor/echo payload: 
2020-03-11 09:59:00.377535 DEBUG home_presence_app: Echo received from unknown: 
2020-03-11 09:59:00.383630 DEBUG home_presence_app: monitor/basement/echo payload: ok
2020-03-11 09:59:00.384714 DEBUG home_presence_app: Echo received from basement: ok
2020-03-11 09:59:00.385324 DEBUG home_presence_app: Canceling timer with handle 70ae1a0ecab14428afdb34213b7ce7bc for home_presence_app
2020-03-11 09:59:00.386267 DEBUG home_presence_app: Registering run_in in 60 seconds for home_presence_app
2020-03-11 09:59:00.394544 DEBUG home_presence_app: monitor/upstairs/echo payload: ok
2020-03-11 09:59:00.395587 DEBUG home_presence_app: Echo received from upstairs: ok
2020-03-11 09:59:00.396159 DEBUG home_presence_app: Canceling timer with handle 3ef2f6d8fdb64f7fa6e9adf799da325d for home_presence_app
2020-03-11 09:59:00.397265 DEBUG home_presence_app: Registering run_in in 60 seconds for home_presence_app
2020-03-11 09:59:00.405165 DEBUG home_presence_app: monitor/lower/echo payload: ok
2020-03-11 09:59:00.406215 DEBUG home_presence_app: Echo received from lower: ok
2020-03-11 09:59:00.406802 DEBUG home_presence_app: Canceling timer with handle a16634609848432cb97a39c1d174330e for home_presence_app
2020-03-11 09:59:00.407723 DEBUG home_presence_app: Registering run_in in 60 seconds for home_presence_app
2020-03-11 09:59:07.434390 DEBUG home_presence_app: monitor/lower/status payload: online
2020-03-11 09:59:07.438558 DEBUG home_presence_app: The Lower Presence System is Online.
2020-03-11 09:59:07.439275 DEBUG home_presence_app: Canceling timer with handle 9d322a18b55e487b872fa558db320ca8 for home_presence_app
2020-03-11 09:59:07.440794 DEBUG home_presence_app: set state: monitor.monitor_state, {'attributes': {'locations': [], 'version': '2.3.0', 'nodes': 4, 'online_nodes': ['Basement', 'Upstairs', 'Lower'], 'offline_nodes': ['Upper'], 'friendly_name': 'Monitor System State'}}
2020-03-11 09:59:07.444082 DEBUG home_presence_app: set state: monitor.lower_state, {'state': 'online', 'attributes': {}}
2020-03-11 09:59:07.453108 DEBUG home_presence_app: monitor/lower/sharon payload: {"id":"__________________","confidence":"0","name":"
Sharon","manufacturer":"Google Inc","type":"KNOWN_MAC","retained":"false","timestamp":"Wed Mar 11 2020 10:59:07 GMT+0200 (IST)","version":"0.2.200"}
2020-03-11 09:59:07.459148 DEBUG home_presence_app: monitor/lower/sharon/rssi payload: -99
2020-03-11 09:59:07.459995 DEBUG home_presence_app: Recieved an RSSI of -99 for sharon from Lower
2020-03-11 09:59:07.460514 DEBUG home_presence_app: set state: monitor.sharon_lower, {'attributes': {'rssi': '-99', 'last_reported_by': 'Lower'}}
2020-03-11 09:59:07.462550 ERROR home_presence_app: Entity sensor.monitor_sharon_lower_conf does not exist.
2020-03-11 09:59:07.465308 DEBUG home_presence_app: update_hass_sensor: Entity_ID: binary_sensor.monitor_sharon, new_state: off
2020-03-11 09:59:07.465978 DEBUG home_presence_app: set state: binary_sensor.monitor_sharon, {'state': 'off', 'attributes': {'device_class': 'presence', 'friendly_name': 'Sharon Home', 'nearest_monitor': 'Unknown', 'type': 'KNOWN_MAC'}}
2020-03-11 09:59:07.480870 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/state', 'payload': '{"locations": [], "version": "2.3.0", "nodes": 4, "online_nodes": ["Basement", "Upstairs", "Lower"], "offline_nodes": ["Upper"], "last_changed": "2020-03-11T09:59:07+01:00", "state": "idle"}'}
2020-03-11 09:59:07.489562 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/lower/state', 'payload': '{"last_rebooted": "", "location": "Lower", "last_changed": "2020-03-11T09:59:07+01:00", "state": "online"}'}

Thanks a lot!

[Odianosen25]

@sharondagan ,

please try the latest dev and see if it helps.

Regards

[sharondagan]

Unfortunately it didn't help.. confidence sensors weren't created.. Maybe I should clear some cache? Cause the previous device succeeded in creating the sensors, but i can't reproduce it anymore

Below are the logs:

2020-03-11 14:48:55.269826 DEBUG home_presence_app: The Lower Presence System is Online.
2020-03-11 14:48:55.270282 DEBUG home_presence_app: Canceling timer with handle 8162c899ae064911bfec182d63ec8825 for home_presence_app
2020-03-11 14:48:55.271429 DEBUG home_presence_app: set state: monitor.monitor_state, {'attributes': {'locations': [], 'version': '2.3.1', 'nodes': 3, 'online_nodes': ['Basement', 'Upstairs', 'Lower'], 'offline_nodes': [], 'friendly_name': 'Monitor System State'}}
2020-03-11 14:48:55.273987 DEBUG home_presence_app: set state: monitor.lower_state, {'state': 'online', 'attributes': {}}
2020-03-11 14:48:55.281342 DEBUG home_presence_app: monitor/lower/rinat payload: {"id":"__________________,"confidence":"0","name":"
Rinat","manufacturer":"LG Electronics Mobile Communications","type":"KNOWN_MAC","retained":"false","timestamp":"Wed Mar 11 2020 15:48:54 GMT+0200 (IST)","version":"0.2.200"}
2020-03-11 14:48:55.287549 DEBUG home_presence_app: monitor/lower/rinat/rssi payload: -99
2020-03-11 14:48:55.288379 DEBUG home_presence_app: Recieved an RSSI of -99 for rinat from Lower
2020-03-11 14:48:55.288865 DEBUG home_presence_app: set state: monitor.rinat_lower, {'attributes': {'rssi': '-99', 'last_reported_by': 'Lower'}}
2020-03-11 14:48:55.289246 WARNING home_presence_app: home_presence_app: Entity monitor.rinat_lower not found in namespace mqtt
2020-03-11 14:48:55.289897 INFO AppDaemon: home_presence_app: Entity monitor.rinat_lower created in namespace: mqtt
2020-03-11 14:48:55.292117 ERROR home_presence_app: Entity sensor.monitor_rinat_lower_conf does not exist, running arrival scan.
2020-03-11 14:48:55.292917 DEBUG home_presence_app: Registering run_in in 0 seconds for home_presence_app
2020-03-11 14:48:55.295946 DEBUG home_presence_app: set state: binary_sensor.monitor_rinat, {'nearest_monitor': 'Unknown'}
2020-03-11 14:48:55.303345 WARNING HASS: Error setting Home Assistant state default.binary_sensor.monitor_rinat, {'nearest_monitor': 'Unknown'}
2020-03-11 14:48:55.303904 WARNING HASS: Code: 400, error: {"message": "No state specified."}
2020-03-11 14:48:55.307016 DEBUG home_presence_app: update_hass_sensor: Entity_ID: binary_sensor.monitor_rinat, new_state: off
2020-03-11 14:48:55.307630 DEBUG home_presence_app: set state: binary_sensor.monitor_rinat, {'state': 'off', 'attributes': {'device_class': 'presence', 'friendly_name': 'Rinat Home', 'nearest_monitor': 'Unknown', 'type': 'KNOWN_MAC'}}
2020-03-11 14:48:55.324777 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/state', 'payload': '{"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Lower"], "offline_nodes": [], "last_changed": "2020-03-11T14:48:55+01:00", "state": "idle"}'}
2020-03-11 14:48:55.332506 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/lower/state', 'payload': '{"last_rebooted": "", "location": "Lower", "last_changed": "2020-03-11T14:48:55+01:00", "state": "online"}'}
2020-03-11 14:48:55.340221 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/scan/arrive', 'payload': ''}
2020-03-11 14:48:55.349498 DEBUG home_presence_app: monitor/state payload: {"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Lower"], "offline_nodes": [], "last_changed": "2020-03-11T14:48:55+01:00", "state": "idle"}
2020-03-11 14:48:55.358020 DEBUG home_presence_app: monitor/lower/state payload: {"last_rebooted": "", "location": "Lower", "last_changed": "2020-03-11T14:48:55+01:00", "state": "online"}

Thank you

[Odianosen25]

Ok give me a few to look into this

[Odianosen25]

Did you try restarting HA? I wouldn't expect that to be a fix, but pls try doing that for now and see if it fixes it. I am getting confused myself why its happening

[sharondagan]

Yeah, I restarted it just now.. no change

[Odianosen25]

Hello @sharondagan,

I just updated the dev with a few extra stuff, Pleas try it. It seems to me that when the entity doesn't exist in HA, but the RSSI is sent first, the system is trying to update it when it doesn't even exist. So I think it has something to do with that.

Please if you update do, restart HA.

Regards

[sharondagan]

well, the good news is there are no errors now... :) However no confidence sensors are created other than the basement ones. I replaced other 2 nodes - livingroom and upstairs, I can see that they're recognized in the log entry below ('online_nodes': ['Basement', 'Upstairs', 'Livingroom']), and I see that monitor sends confidence report via the mqtt (to monitor/livingroom/sharon topic). But nothing is changed on hass UI

2020-03-11 18:30:03.647432 DEBUG home_presence_app: monitor/basement/status payload: online
2020-03-11 18:30:03.648352 DEBUG home_presence_app: The Basement Presence System is Online.
2020-03-11 18:30:03.648814 DEBUG home_presence_app: Canceling timer with handle 11c99828a2fe477eb07f80b5195a068e for home_presence_app
2020-03-11 18:30:03.649981 DEBUG home_presence_app: set state: monitor.monitor_state, {'attributes': {'locations': [], 'version': '2.3.1', 'nodes': 3, 'online_nodes': ['Basement', 'Upstairs', 'Livingroom'], 'offline_nodes': [], 'friendly_name': 'Monitor System State'}}
2020-03-11 18:30:03.652582 DEBUG home_presence_app: set state: monitor.basement_state, {'state': 'online', 'attributes': {}}
2020-03-11 18:30:03.660095 DEBUG home_presence_app: monitor/basement/sharon payload: {"id":"3C:28:6D:03:63:F3","confidence":"0","name":"Sharon","manufacturer":"Unknown","type":"KNOWN_MAC","retained":"false","timestamp":"Wed Mar 11 2020 19:30:03 GMT+0200 (IST)","version":"0.2.200"}
2020-03-11 18:30:03.662842 DEBUG home_presence_app: update_hass_sensor: Entity_ID: sensor.monitor_sharon_basement_conf, new_state: 0
2020-03-11 18:30:03.663442 DEBUG home_presence_app: set state: sensor.monitor_sharon_basement_conf, {'state': 0, 'attributes': {'friendly_name': 'Sharon Basement', 'id': '3C:28:6D:03:63:F3', 'location': 'basement', 'manufacturer': 'Unknown', 'name': 'Sharon', 'retained': 'false', 'rssi': '-99', 'timestamp': 'Wed Mar 11 2020 19:30:03 GMT+0200 (IST)', 'type': 'KNOWN_MAC', 'unit_of_measurement': '%', 'version': '0.2.200'}}
2020-03-11 18:30:03.678526 DEBUG home_presence_app: set state: monitor.sharon_basement, {'state': 0, 'attributes': {'id': '3C:28:6D:03:63:F3', 'name': 'Sharon', 'manufacturer': 'Unknown', 'type': 'KNOWN_MAC', 'retained': 'false', 'timestamp': 'Wed Mar 11 2020 19:30:03 GMT+0200 (IST)', 'version': '0.2.200', 'friendly_name': 'Sharon Basement', 'location': 'basement'}}
2020-03-11 18:30:03.688463 DEBUG home_presence_app: monitor/basement/sharon/rssi payload: -99
2020-03-11 18:30:03.689699 DEBUG home_presence_app: Recieved an RSSI of -99 for sharon from Basement
2020-03-11 18:30:03.690883 DEBUG home_presence_app: set state: monitor.sharon_basement, {'attributes': {'rssi': '-99', 'last_reported_by': 'Basement'}}
2020-03-11 18:30:03.694664 DEBUG home_presence_app: update_hass_sensor: Entity_ID: sensor.monitor_sharon_basement_conf, new_state: 0
2020-03-11 18:30:03.695330 DEBUG home_presence_app: set state: sensor.monitor_sharon_basement_conf, {'state': '0', 'attributes': {'friendly_name': 'Sharon Basement', 'id': '3C:28:6D:03:63:F3', 'location': 'basement', 'manufacturer': 'Unknown', 'name': 'Sharon', 'retained': 'false', 'rssi': '-99', 'timestamp': 'Wed Mar 11 2020 19:30:03 GMT+0200 (IST)', 'type': 'KNOWN_MAC', 'unit_of_measurement': '%', 'version': '0.2.200'}}
2020-03-11 18:30:03.705945 DEBUG home_presence_app: set state: binary_sensor.monitor_sharon, {'nearest_monitor': 'Unknown'}
2020-03-11 18:30:03.713175 WARNING HASS: Error setting Home Assistant state default.binary_sensor.monitor_sharon, {'nearest_monitor': 'Unknown'}
2020-03-11 18:30:03.713699 WARNING HASS: Code: 400, error: {"message": "No state specified."}
2020-03-11 18:30:03.716660 DEBUG home_presence_app: update_hass_sensor: Entity_ID: binary_sensor.monitor_sharon, new_state: off
2020-03-11 18:30:03.717206 DEBUG home_presence_app: set state: binary_sensor.monitor_sharon, {'state': 'off', 'attributes': {'device_class': 'presence', 'friendly_name': 'Sharon Home', 'nearest_monitor': 'Unknown', 'type': 'KNOWN_MAC'}}
2020-03-11 18:30:03.731190 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/state', 'payload': '{"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Livingroom"], "offline_nodes": [], "last_changed": "2020-03-11T18:30:03+01:00", "state": "idle"}'}
2020-03-11 18:30:03.739354 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/basement/state', 'payload': '{"last_rebooted": "", "location": "Basement", "last_changed": "2020-03-11T18:30:03+01:00", "state": "online"}'}
2020-03-11 18:30:03.751071 DEBUG home_presence_app: monitor/state payload: {"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Livingroom"], "offline_nodes": [], "last_changed": "2020-03-11T18:30:03+01:00", "state": "idle"}
2020-03-11 18:30:03.757675 DEBUG home_presence_app: monitor/basement/state payload: {"last_rebooted": "", "location": "Basement", "last_changed": "2020-03-11T18:30:03+01:00", "state": "online"}

[Odianosen25]

Phew glad we getting somewhere 😅. Ok I have seen another bug which is similar to the previous one, but on the device the issue is that for some reasons they not being picked up by the nodes a really. From the logs, I can’t seem to find any indication the nodes sent data from that location.

Can you kindly log into say the living room one, confirm the known devices are in the file as entered in the app config. Now if confirmed, can you use a standalone MQTT client to check if data is actually being sent from those nodes, so I am sure it’s not an AD issue as a whole.

Regards

[sharondagan]

:)

ok - just validated that the nodes are reporting statuses. Also, I was able to see the sent messages using mqtt explorer.

2020-03-11 19:36:16.606349 DEBUG home_presence_app: monitor/upstairs/status payload: online
2020-03-11 19:36:16.607603 DEBUG home_presence_app: The Upstairs Presence System is Online.
2020-03-11 19:36:16.608345 DEBUG home_presence_app: Canceling timer with handle fca2d4a151534d8eb197a5b373643b12 for home_presence_app
2020-03-11 19:36:16.609718 DEBUG home_presence_app: set state: monitor.monitor_state, {'attributes': {'locations': [], 'version': '2.3.1', 'nodes': 3, 'online_nodes': ['Basement', 'Upstairs', 'Livingroom'], 'offline_nodes': [], 'friendly_name': 'Monitor System State'}}
2020-03-11 19:36:16.613034 DEBUG home_presence_app: set state: monitor.upstairs_state, {'state': 'online', 'attributes': {}}
2020-03-11 19:36:16.622597 DEBUG home_presence_app: monitor/upstairs/sharon payload: {"id":"3C:28:6D:03:63:F3","confidence":"0","name":"
Sharon","manufacturer":"Unknown","type":"KNOWN_MAC","retained":"false","timestamp":"Wed Mar 11 2020 20:36:16 GMT+0200 (IST)","version":"0.2.200"}
2020-03-11 19:36:16.629188 DEBUG home_presence_app: monitor/upstairs/sharon/rssi payload: -99
2020-03-11 19:36:16.630074 DEBUG home_presence_app: Recieved an RSSI of -99 for sharon from Upstairs
2020-03-11 19:36:16.638431 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/state', 'payload': '{"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Livingroom"], "offline_nodes": [], "last_changed": "2020-03-11T19:36:16+01:00", "state": "idle"}'}
2020-03-11 19:36:16.648839 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/basement/state', 'payload': '{"last_rebooted": "", "location": "Basement", "reboot_scheduled": false, "reboot_time": "2020-03-12T02:00:01", "last_changed": "2020-03-11T19:36:16+01:00", "state": "online"}'}
2020-03-11 19:36:16.659500 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/state', 'payload': '{"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Livingroom"], "offline_nodes": [], "last_changed": "2020-03-11T19:36:16+01:00", "state": "idle"}'}
2020-03-11 19:36:16.667395 DEBUG home_presence_app: call_service: mqtt/publish, {'topic': 'monitor/upstairs/state', 'payload': '{"last_rebooted": "", "location": "Upstairs", "reboot_scheduled": false, "reboot_time": "2020-03-12T02:00:01", "last_changed": "2020-03-11T19:36:16+01:00", "state": "online"}'}
2020-03-11 19:36:16.675320 DEBUG home_presence_app: monitor/state payload: {"locations": [], "version": "2.3.1", "nodes": 3, "online_nodes": ["Basement", "Upstairs", "Livingroom"], "offline_nodes": [], "last_changed": "2020-03-11T19:36:16+01:00", "state": "idle"}
2020-03-11 19:36:16.681028 DEBUG home_presence_app: monitor/basement/state payload: {"last_rebooted": "", "location": "Basement", "reboot_scheduled": false, "reboot_time": "2020-03-12T02:00:01", "last_changed": "2020-03-11T19:36:16+01:00", "state": "online"}

[Odianosen25]

Ok based on this logs, can you confirm if sharon has confidence sensors for both upstairs and basement?

[sharondagan]

yes

[sharondagan]

Found it!!! the bluetooth mac addresses that are automatically populated to the monitor nodes are upper cased. I manually changed it to lower case, and the sensors were generated

[Odianosen25]

Uh? You shouldn't need to do that really. I still think there is something abt how the device is sensed by the nodes that's messing with it. Let me fix a bug I saw and upload, and take it up from there

[Odianosen25]

Ok @sharondagan,

I have fixed some bug s I had created earlier, and pls use this new version and I advise you revert back to using the upper case Mac address as before.

Also when you download this new version, kindly restart HA so it clears some bad actors entities I might have created within it, which was possibly what was causing your issue.

This could explain why when you edited the MAC, it fixed it. I have a different setup in that my default namespaces is free and my HA namespaces is hass. So when doing some stuffs in my environment, using default to manipulate data is fine. But I forgot most users actually have HA on default so messing with the setup.

Apologies for that. Do let me know how it goes.

Regards

[sharondagan]

thanks for the fix!

So - I've erased the entities on known_static_addresses. They were re-created upper cased. I can see the basement & upstairs confidence sensors, but not livingroom 🤔 (i've validated that this node sends statuses)

Here's a screenshot of appdaemon's mqtt

[Odianosen25]

Well the fact it sends status, doesn't mean it sees the phone to report the phone's confidence. The only way to confirm it sends confidence, is to use the separate mqtt client to check for if data is sent from that location for the device in question.

Regards

[sharondagan]

I'll summarize - now it works very well, but only when mac addresses (in known_static_addresses) are lower cased for some reason 🤔 Setting them uppercase causes the sensors not to be generated when hass loads.

[Odianosen25]

Hello @sharondagan,

Thanks for the update and that's good to hear. This issue has to do with the nodes, but I will advise you try one more thing.

Clear the presently known devices in all your nodes, as I think you will have both upper and lower case MAC addresses in them, and it might be messing with the system's head. Then try using the upper case again so it loads it into all the nodes at the same time.

Hopefully this will completely fix the issues you having, of having to use lower case instead of upper.

Regards

[sharondagan]

Thanks for replying! I tried exactly what you instructed (although I had only one variance - lowercase), unfortunately it didn't help 😞-- it created the entries again, uppercased. After a restart of HA, it didn't show. I know - it's super weird..

just a guess - maybe HA persisted the uppedcased sensor variance somewhere, and now it can't create it?