[Feature request] XG3 offline device - follow this status in HA

Masterz69 commented 3 years ago

Home Assistant OS 5.12, core-2021.3.0.

ZNDMWG02LM, lumi.sen_ill.mgl01 GZCGQ01LM

After 2 hours sensor become OFFLINE in Xiaomi Home application. Screenshot_20210307-163521_Mi Home

But related entities in HA still have no state 'unavailable'.

IMHO as in, f.e., Sonoff integration - entities, related to offline devices, become "unavailable".

Masterz69 commented 3 years ago

2021-03-09 08:58:08 DEBUG gateway3 192.168.1.16 | Get devices 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | lumi.158d00056e6606 lumi.weather.v1 retain: {'CCA': 1, 'alive': 1, 'battery': 67, 'battery_end_of_life': 0, 'chip_temperature': 0, 'cur_state': 3, 'fw_ver': 0, 'humidity': 2636, 'hw_ver': 0, 'lqi': 0, 'power_tx': 10, 'pre_state': 1, 'pressure': 101890, 'pv_state': 0, 'reset_cnt': 66, 'send_all_cnt': 1, 'send_fail_cnt': 0, 'send_retry_cnt': 0, 'temperature': 2211, 'voltage': 3005} 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | lumi.4cf8cdf3c78a962 lumi.sen_ill.mgl01 retain: {'2.1': 341, '3.1': 3100, 'alive': 1} 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.0', 'model': 'lumi.gateway.mgl03', 'mac': '0x842e14fffe4df3ca', 'type': 'gateway', 'init': {'firmware lock': True, 'alarm_did': '374526720'}} 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.158d00056e6606', 'mac': '0x158d00056e6606', 'ieee': '00158d00056e6606', 'nwk': '0e00', 'model': 'lumi.weather.v1', 'type': 'zigbee', 'fw_ver': 0, 'init': {'temperature': 22.11, 'humidity': 26.36, 'pressure': 1018.9, 'battery': 67}, 'online': True} 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.4cf8cdf3c78a962', 'mac': '0x4cf8cdf3c78a962', 'ieee': '04cf8cdf3c78a962', 'nwk': '9ffd', 'model': 'lumi.sen_ill.mgl01', 'type': 'zigbee', 'fw_ver': None, 'init': {'illuminance': 341, 'battery': 86}, 'online': True} 2021-03-09 08:58:09 DEBUG gateway3 192.168.1.16 | Gateway time offset: -0.4861137866973877

**^^ see three devices in ZigBee network, inl. XG3

09:00 - sensor battery removed, last message in debug:**

2021-03-09 09:00:47 DEBUG gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/MessageReceived b'{"sourceAddress":"0x9FFD","eui64":"0x04CF8CDF3C78A962","destinationEndpoint":"0x01","clusterId":"0x0400","profileId":"0x0104","sourceEndpoint":"0x01","APSCounter":"0x4D","APSPlayload":"0x18CB0A000021C62C","rssi":-62,"linkQuality":152}' 2021-03-09 09:00:47 DEBUG gateway3 192.168.1.16 | MQTT | zigbee/send b'{"cmd":"report","id":2000001090,"did":"lumi.4cf8cdf3c78a962","dev_src":"0","time":1615273246979,"rssi":-62,"mi_spec":[{"siid":2,"piid":1,"value":13}]}' 2021-03-09 09:00:47 DEBUG gateway3 192.168.1.16 | lumi.4cf8cdf3c78a962 lumi.sen_ill.mgl01 <= {'illuminance': 13} [0.0] ... 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"debugprint all_on"}' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"plugin device-table print"}' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/miio b'[20210309 10:29:29] [D] ot_agent_recv_handler_one(): fd:12, msg:{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":52562}{"method":"props","model":"lumi.weather.v1","params":{"humidity":2640},"sid":"lumi.158d00056e6606","id":52563}{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":52564}{"method":"props","model":"lumi.weather.v1","params":{"pressure":101862},"sid":"lumi.158d00056e6606","id":52565}{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":52566}{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":52567} length:618 bytes' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'0 9FFD: 04CF8CDF3C78A962 0 JOINED 5323\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'1 0E00: 00158D00056E6606 0 JOINED 0\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'Total Devices 2\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: plugin device-table print\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'# type id eui64 timeout\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'0: Sleepy 0x9FFD (>)04CF8CDF3C78A962 512 min debug timeout:249\r' 2021-03-09 10:29:30 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'1: Sleepy 0x0E00 (>)00158D00056E6606 512 min debug timeout:249\r'

**^^ sensor still ONLINE in MiHome & HA.

11:15 - sensor appears OFFLINE in MiHome. Unfortuantely see no message in debug log from XG3 - about sensor becoming offline :(**

2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG 2021-03-09 11:48:59 DEBUG ... 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG 2021-03-09 11:49:00 DEBUG gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: debugprint all_on\r' gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"debugprint all_on"}' gateway3 192.168.1.16 | MQTT | log/miio b'[20210309 11:48:58] [D] ot_agent_recv_handler_one(): fd:12, msg:{"method":"props","model":"lumi.weather.v1","params":{"device_log":"[1615283338,[\"prop.temperature\",[2217]]]"},"sid":"lumi.158d00056e6606","id":53392}{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":53393}{"method":"event.keepalive","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":53394}{"method":"props","model":"lumi.weather.v1","params":{"humidity":2631},"sid":"lumi.158d00056e6606","id":53395}{"method":"props","model":"lumi.weather.v1","params":{"device_log":"[1615283338,[\"prop.humidity\",[2631]]]"},"sid":"lumi.158d00056e6606","id":53396}{"method":"event.dry","model":"lumi.weather.v1","params":[],"sid":"lumi.158d00056e6606","id":53397} length:714 bytes' gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"plugin device-table print"}' gateway3 192.168.1.16 | MQTT | log/z3 b'0 9FFD: 04CF8CDF3C78A962 0 JOINED 10092\r' gateway3 192.168.1.16 | MQTT | log/z3 b'1 0E00: 00158D00056E6606 0 JOINED 0\r' gateway3 192.168.1.16 | MQTT | log/z3 b'Total Devices 2\r' gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: plugin device-table print\r' gateway3 192.168.1.16 | MQTT | log/z3 b'\r' gateway3 192.168.1.16 | MQTT | log/z3 b'# type id eui64 timeout\r' gateway3 192.168.1.16 | MQTT | log/z3 b'0: Sleepy 0x9FFD (>)04CF8CDF3C78A962 512 min debug timeout:249\r' gateway3 192.168.1.16 | MQTT | log/z3 b'1: Sleepy 0x0E00 (>)00158D00056E6606 512 min debug timeout:249\r' gateway3 192.168.1.16 | MQTT | log/z3 b'' gateway3 192.168.1.16 | MQTT | log/z3 b'2 of 32 entries used.\r' gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"plugin stack-diagnostics child-table"}' gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: plugin stack-diagnostics child-table\r' gateway3 192.168.1.16 | MQTT | log/z3 b'' gateway3 192.168.1.16 | MQTT | log/z3 b'# id lqi in out age eui\r' gateway3 192.168.1.16 | MQTT | log/z3 b'' gateway3 192.168.1.16 | MQTT | log/z3 b'0 of 26 entries used.\r' gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"plugin stack-diagnostics neighbor-table"}' gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: plugin stack-diagnostics neighbor-table\r' gateway3 192.168.1.16 | MQTT | log/z3 b'0: 0x9FFD -> 0x0000 (Me)\r' gateway3 192.168.1.16 | MQTT | log/z3 b'1: 0x0E00 -> 0x0000 (Me)\r' gateway3 192.168.1.16 | MQTT | log/z3 b'2 of 255 total entries.\r' gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"plugin concentrator print-table"}' gateway3 192.168.1.16 | MQTT | log/z3 b'CLI command executed: plugin concentrator print-table\r' gateway3 192.168.1.16 | Update parent info table gateway3 192.168.1.16 | Total zigbee devices: 2 gateway3 192.168.1.16 | MQTT | gw/842E14FFFE4DF3CA/executed b'{"command":"debugprint all_off"}'

**After 3 hours no data received in HA - sensor and entities still ONLINE.

IMHO this is not good/right. Please find a way to detect sensor/device OFFLINE status.**

Masterz69 commented 3 years ago

Test performed on 2 devices: Xiaomi Lumi GZCGQ01LM & Aqara THP WSDCGQ11LM. After 2 hours bth become Offline in MiHome. In HA - see 'unknown' on entities, all entities not grayed (NOT unavailable ?). 2 hours debug log attached.

xg3_sensor_offline_20210311.zip xg3_sensor_offline_20210311_01 xg3_sensor_offline_20210311_02 xg3_sensor_offline_20210311_03 ^^ %Living Room% - sensors from XG3.

Masterz69 commented 3 years ago

HA restarted while sensors OFFLINE. See attached log and screenshot from MiHome.

xg3_sensor_offline_20210311_ha_restarted.zip

Screenshot_20210311-121903_Mi Home

Masterz69 commented 3 years ago

2021-03-11 12:16:32 DEBUG gateway3 192.168.1.16 | Get devices 2021-03-11 12:16:32 DEBUG gateway3 192.168.1.16 | lumi.158d00056e6606 lumi.weather.v1 retain: {'CCA': 1, 'alive': 1, 'battery': 65, 'battery_end_of_life': 0, 'chip_temperature': 0, 'cur_state': 3, 'fw_ver': 0, 'humidity': 3154, 'hw_ver': 0, 'lqi': 0, 'power_tx': 10, 'pre_state': 1, 'pressure': 101486, 'pv_state': 0, 'reset_cnt': 66, 'send_all_cnt': 0, 'send_fail_cnt': 0, 'send_retry_cnt': 0, 'temperature': 2052, 'voltage': 2995} 2021-03-11 12:16:32 DEBUG gateway3 192.168.1.16 | lumi.4cf8cdf3c78a962 lumi.sen_ill.mgl01 retain: {'2.1': 3, '3.1': 3100, 'alive': 1} 2021-03-11 12:16:34 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.0', 'model': 'lumi.gateway.mgl03', 'mac': '0x842e14fffe4df3ca', 'type': 'gateway', 'init': {'firmware lock': True, 'alarm_did': '374526720'}} 2021-03-11 12:16:37 DEBUG main Loaded from MiCloud 3 devices 2021-03-11 12:16:37 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.158d00056e6606', 'mac': '0x158d00056e6606', 'ieee': '00158d00056e6606', 'nwk': '0e00', 'model': 'lumi.weather.v1', 'type': 'zigbee', 'fw_ver': 0, 'init': {'temperature': 20.52, 'humidity': 31.54, 'pressure': 1014.86, 'battery': 65}, 'online': True} 2021-03-11 12:16:37 DEBUG gateway3 192.168.1.16 | Setup Zigbee device {'did': 'lumi.4cf8cdf3c78a962', 'mac': '0x4cf8cdf3c78a962', 'ieee': '04cf8cdf3c78a962', 'nwk': '9ffd', 'model': 'lumi.sen_ill.mgl01', 'type': 'zigbee', 'fw_ver': None, 'init': {'illuminance': 3, 'battery': 86}, 'online': True} 2021-03-11 12:16:37 DEBUG gateway3 192.168.1.16 | Gateway time offset: -0.7612943649291992

^^ see online : True & alive :1 , BUT both devices have OFFLINE status in Mihome - see screenshot in previous post.

Is it mean - no alive:0 message(s) sent by XG3 and HA getting last/retained message...?

Masterz69 commented 3 years ago

Proceeded 2nd HA core restart - thanx SonoffLAN new version :)

Now see alive:0 for one of two sensors, both w/out batteries last 3 hours.

xg3_sensor_offline_20210311_ha_restarted_2.txt

See no Unavailable status in entities list: xg3_sensor_offline_20210311_04

SEE in dashboard: xg3_sensor_offline_20210311_05

Masterz69 commented 3 years ago

As discussed with AlexxIT - logs clearly show following behaviour (at least for that two kind of sensors):

MiHome claim disconnected sensor as OFFLINE after ~2+ hours;
information to outer recipients, like HA - sent after 3+ hours.

AlexxIT / XiaomiGateway3