zigbee map - devices work but don't show as connected

[h4nc]

My zigbee map (zigbee component) already worked perfectly. Devices always showed the connection.

I recently decided to make my network more secure and add a network key. As you know this means re-pairing all the devices. So I also updated the firmware of my coordinator and of my routers.

I'm using the edge addon right now, but 5 of my 14 devices don't show a connection in the map. But all of them work. It's not an issue of the map component, because the info about the connection isn't present at the code either.

This is a device that doesn't show it's connection: "0xID1" [style="rounded, dashed", label="{Zigbee Button|EndDevice|Xiaomi Aqara wireless switch (WXKG11LM)|online}"];

This is one that shoes the linkqualitiy (50): "0xID2" [style="rounded, dashed", label="{Zigbee Fenster SZ|EndDevice|Xiaomi Aqara door & window contact sensor (MCCGQ11LM)|online}"]; "0xID2" -> "0xID_COORD" [label="50"]

So the second line is missing. Do I have to delte or reset something in the database or state.json to make this work again.

The devices are all at the same place as before. My routers are connected to the coord.

[dh-harald]

Unfortunately, I've got ping timeout for QBKG11LM and QBKG04LM as well...

[h4nc]

I also get this error lately

zigbee2mqtt:error 3/5/2019, 7:59:15 PM Failed to ping 0x00XXXX

It's CC2530 flashed as a router.

[Koenkk]

Can somebody do some sniffing to find out what is going on? (http://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html)

[h4nc]

I moved the router nearer to the coordinator but I still get the "failed to ping" message.

I have the new router firmware, where you can reset it by unplugging it 5 times. I will try to re-pair it.

EDIT: These messages are gone after I reflashed that one router that made issues.

[h4nc]

I think I found a solution for this issue. Maybe someone here that still has the issue can try that and tell if it worked.

I already wrote this here: https://github.com/Koenkk/zigbee2mqtt/issues/1235#issuecomment-471532387

Again this brought up the issue where all devices did not show a connection in the zigbee map. I was able so solve this this way:

• Set permit_join to true

• unplug and plug in your routers.

I did both of those options, so I cant says if both are necessary or only one is ok.

EDIT: After the steps above I noticed that some of my devices show a much better linkquality because they picked a router instead of the coordinator. A device that only had a linkquality of 1 (but worked) has 150 now, because it picked a nearer router. Before nearly all devices where connected directly to the coordinator in the zigbee map. @Koenkk Aren't those enddevices supposed to pick the best router/coord by it's own? How does it work? Also only one enddevice is connected to the coord now (information from the zigbee map). This device could have a better linkqualtity if it also chooses the much nearer router. Is it possible to force the devices to use a certain router? Or is this simple a mistake in the map and it already uses the best connection?

[dh-harald]

@h4nc: I'm playing with this for a long time... I'm always re-pairing the routers. Yesterday, I've flashed my coordinator to the latest firmware, upgraded zigbee2mqtt to 1.2.1. It look worked for a while, The routers connected each other, instead of controller... I could restart the controller, without messing the topology... I've got ping timeout messages, but it looks really good... The linkqualities was high... I was really happy... But after couple of minutes, everything went back to "normal"... Every router is now connected to the controller... I've ordered another stick, I'll try to sniff the network for @Koenkk I hope, He can solve this problem, because my network looks quiet unhealthy... this is my favourite:

zigbee2mqtt:info 3/10/2019, 9:37:26 PM MQTT publish: topic 'zigbee2mqtt/light_corridor', payload '{"power":41.9,"linkquality":70,"consumption":0.51,"temperature":27,"state":"ON"}'
zigbee2mqtt:error 3/10/2019, 9:37:26 PM Failed to ping << device ^^^^ >>
zigbee2mqtt:info 3/10/2019, 9:37:39 PM MQTT publish: topic 'zigbee2mqtt/light_corridor', payload '{"power":41.9,"linkquality":73,"consumption":0.52,"temperature":28,"state":"ON"}'

[h4nc]

Did you try the two steps I wrote about in my last post?

[dh-harald]

No, usually, I'm repairing them... I'll try it tonight

[dh-harald]

But it's not my main problem... Even if I create a proper network map (sort of), I've got ping timeout messages from the routers, and for some unknown reason my hardly built topology falls back to the well known star topology (every router connected only to the controller). I think, the ping timeouts and the topology fallback is connected somehow

[h4nc]

I also had both issues. The two steps fixed them for me.

Now it would be interesting if that was a coincidence or it solves the issues for you and others too.

As I said and don't know if both steps are even necessary. Maybe only one of those, do the trick.

[dh-harald]

@h4nc it didn't worked for me...

[dh-harald]

@Koenkk I've sent you network sniff to your email address

[Koenkk]

@dh-harald it's hard to determine such thing from the sniffing log, what are you problems, only the pining problems?

[dh-harald]

@Koenkk My main problem, that I describe, that the zigbee network looks unhealty. Even is, the routers are think, that they're online, the controller pings are timing out all the time... The availibility stuff is not working well, because it's putting the device offline. Here's a really good log from my zigbee2mqtt logs

zigbee2mqtt:info 3/10/2019, 9:37:26 PM MQTT publish: topic 'zigbee2mqtt/light_corridor', payload '{"power":41.9,"linkquality":70,"consumption":0.51,"temperature":27,"state":"ON"}'
zigbee2mqtt:error 3/10/2019, 9:37:26 PM Failed to ping << device ^^^^ >>
zigbee2mqtt:info 3/10/2019, 9:37:39 PM MQTT publish: topic 'zigbee2mqtt/light_corridor', payload '{"power":41.9,"linkquality":73,"consumption":0.52,"temperature":28,"state":"ON"}'

same as like https://github.com/zigbeer/zigbee-shepherd/issues/8 or https://github.com/Koenkk/zigbee2mqtt/issues/1248

Tonight, I've made a test with moving the controller really close (1m) to the router (wall switch - QBKG11LM) and I've started pairing the devices... The link quality was 120+, and after a couple of connected devices, the whole process has been started, as I described earlier:

• The routers are connected only with the controller, not each other (as the network map shows)
• The controller can't ping the routers (mainly QBKG11LM), even if the distant is less then 1m, and the link quality > 120 which published into mqtt

[Koenkk]

• The networkmap shouldn't be used to determine if your zigbee networkmap is healthy or not. It's still incomplete (and may never be complete) because:
  • There are probably still bugs on the zigbee2mqtt side
  • It depends on the device functionality. If it will ever work properly or not also depends on the devices in your network. If some of them don't pass the correct information, you will never get a complete networkmap.
  • Enddevices could be sleeping while the network scan is executed, therefore they cannot respond and will show as disconnected (while in practice they are not!)
• My personal view on the networkmap is that it can assist users to find problematic parts in their zigbee network, it shouldn't be used as the single source of truth (!!). Pursuing a complete interconnected networkmap is useless, if your zigbee network works properly, you shouldn't care about the networkmap.
• About the pining part, zigbee2mqtt will send a certain zigbee command to a device, if the device doesn't respond, zigbee2mqtt marks it as failed. So again, if the device is connected, but for some reason just doesn't respond (e.g due to a crappy zigbee implementation on the device or due to routers dropping this packet), this pining mechanism won't work. In practice you will e.g. see that the device still responds to on/off commands.
• Given that your linkquality is so high and that the device is so closely located to the coordinator, I don't expect that the zigbee network is the issue here. It would be interesting to retrieve a zigbee sniffing log of +- 10 seconds where at second 0 the ping request is send. In this way we can verify if the device responds or not.

[h4nc]

My personal view on the networkmap is that it can assist users to find problematic parts in their zigbee network, it shouldn't be used as the single source of truth (!!). Pursuing a complete interconnected networkmap is useless, if your zigbee network works properly, you shouldn't care about the networkmap.

I think the thing is most users really like to see there devices in their maps. The maps are getting bigger with every device and it's nice to have a visualization. The issue here is (that most of us users (me too) forget or don't know that the map could be faulty) and take it as bulletproof.

I also think that a lot of us already use this component https://github.com/rgruebel/ha_zigbee2mqtt_networkmap

Maybe it would be a good idea to place a watermark somewhere on that page to tell users that this is beta (or something else that tells you that it isn't a 100% reliable). What do you think about that @rgruebel

[dh-harald]

I'm not using the networkmap to determine is my zigbee network healthy or not. I can understand, that my crappy implemented zigbee switch isn't responding to the pings, but I think, there's a lot of stuff can trigger based on this response... For example the availability is, because it's deconfiguring the switch -> mqtt light, so even if it should work, I can't because it's not available... Another thing, that this is my guess as well, and this based on the network map: I think, if the router can't be ping, it triggers some kind of network topology change in the controller. That's why I said, that i could create a "good" topology, but sometimes after a failed ping, the network topology is changing to that "star" variant, when the routers are connected only to the controller... I don't know, is it a valid network map or not, I can't get that info other than the network map, that zigbee2mqtt generates.... I saw, that you've implemented, that I could disable availability check for devices, which is cool, thanks for that... I've an unboxed Xiaomi gateway as well, I bought it before I've found your project, I can make tests if it helps you, or I can send it to you if you have QBKG11LM to debug it's crappy zigbee implementation.... I'm really like your project, it's amazing, but this issue drives me mad for a long time now :(

[Koenkk]

• For devices which don't respond to ping request, as you mentioned, you can use the disable availability check. (for now, as a workaround)
• For pinging issues with Xiaomi devices, let's continue in #1248
• Good that you mention the Xiaomi gateway, I also have one, let's check how they do it.

[h4nc]

„crappy implemented zigbee switch“

Why is it crappy implemented, I’m totally thrilled by the quality of aqara/Xiaomi zigbee devices. They all work perfect. However I do that own that certain switch.

[Koenkk]

They don't follow the ZCL spec. (e.g. data is in a proprietary format).

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.