search

Koenkk / zigbee2mqtt

Zigbee 🐝 to MQTT bridge 🌉, get rid of your proprietary Zigbee bridges 🔨
https://www.zigbee2mqtt.io
GNU General Public License v3.0
12.13k stars 1.68k forks source link

Z-Stack 3 on CC2531/CC2530 adventures #1445

Closed Koenkk closed 4 years ago

Koenkk commented 5 years ago

This issue is created in order to gather feedback of users using the CC2531/CC2530 AND Z-Stack 3.0 (= Zigbee 3.0)

You are about to enter a highly experimental area (no guarantees)

Notes

Known issues

Koenkk commented 5 years ago

@dzungpv I'm currently still investigating the CC2652R issues. CC2538 might also be an option #1568

dzungpv commented 5 years ago

@Koenkk I only try with CC2530/2531, but i have some problem with z2mqtt, every time restart z2mqtt it will start pairing from the start, so it is about z2mqtt? I read some one use Ztool solution to start the Z 3.0 network and it work stable.

Koenkk commented 5 years ago

@dzungpv you mean that all your pairing are lost?

dzungpv commented 5 years ago

@Koenkk Yes, and it start re-pairing all the devices, I setup test network with only 1 router and two end devices

Koenkk commented 5 years ago

@dzungpv strange, I currently use the CC2531 + zigbee 3 in production and it's working reliable for 2 months now.

Probably it zigbee-herdsman detect that the configuration is not up-to-date and therefore reinitialises the device, can you provide the log when running with DEBUG=* npm start?

dzungpv commented 5 years ago

@Koenkk I connect 22 device in the network, 2 router and it start error when reboot/start and it never work again unless i re flash, but it only work the first time and repeat:

  zigbee2mqtt:error 8/9/2019, 8:08:25 PM 
    {"message":"Cannot read property 'zclClustersReq' of null","stack":"TypeError: Cannot read property 'zclClustersReq' of null\n    at /zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/dist/lib/components/event_handlers.js:243:32\n    at _fulfilled (/zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:854:54)\n    at /zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:883:30\n    at Promise.promise.promiseDispatch (/zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:816:13)\n    at /zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:877:14\n    at runSingle (/zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:137:13)\n    at flush (/zigbee2mqtt-1.5.1/node_modules/zigbee-herdsman/node_modules/q/q.js:125:13)\n    at process._tickCallback (internal/process/next_tick.js:61:11)"}
dzungpv commented 5 years ago

@Koenkk never mind, i delete all and start over, now it work fine for 48 hours, 2 CC2530 Z-Stack 3 router with 24 end devices on CC2530+CC2591, include those Xiaomi sensors: 2 motion, 3 temperature, 1 door. I will report back it not stable.

choeflake commented 5 years ago

@Koenkk I could not find any note about a 40 device limit in Zigbee 3. Is there a device limit in combination with the CC2530/31 and Zigbee 3 or general for Zigbee 3? It's hard for me to believe that is is a general device limit in Zigbee 3.

Koenkk commented 5 years ago

Only in combination with the cc2531/cc2530.

antst commented 5 years ago

I found that with current implementation (where you don't set panId at all) it is a bit of lottery with unpredictable results (I tested 3.0.2 on cc2538 though), and while it works for some time, under some condition it can stop (giving every time different panid after stick power cycle ). But, on 3.0.2 you actually CAN set PanId and extPanId from shepherds and it works. I patched code and I have now panid and extpanid I request from config file. If I change panid in config and restart - I get another one. Fully correct behaviour. This is patch for herdsman 0.5.7 (one is installed currently with my dev branch). I wanted to make RP, but realised that in current dev of herdsman things are completely different.

panId2.patch.zip

antst commented 5 years ago

By the way, all those reports about lost pairing, very likely, due to floating PAN ID. This is what I see. I didn't have this problem on my sticks, but probably because of infinite chain of reflashes I flashed something somewhere which made it stable (I was not erasing chip), but when I started to experiment, with erasing chip etc, I ended up in situation when firmware which was stable for months, now also gives me different pan_id on every reboot. And, generally problem shown to be tricky. It can go unchanged for 10 reboots, and start to change on 11th, and then after another 20 reboots become stable back. And it possibly depends on how long boot takes. Apparently this issue was reported to TI year ago, and TI did nothing so far. Patch above fixed it by restoring behaviour which is current with 1.2 stack, It correctly set pan_id on z2m start.

this was original thread about issue http://e2e.ti.com/support/wireless-connectivity/zigbee-and-thread/f/158/t/746836

but today's testing (we did over 400 reboots/power cycles) showed that there is more story to it than just ZDSECMGR_TC_DEVICE_MAX.

antst commented 5 years ago

It looks like there is a bug in binary-only part of z-stack.

Koenkk commented 5 years ago

I found that with current implementation (where you don't set panId at all) it is a bit of lottery with unpredictable results (I tested 3.0.2 on cc2538 though), and while it works for some time, under some condition it can stop (giving every time different panid after stick power cycle ). But, on 3.0.2 you actually CAN set PanId and extPanId from shepherds and it works. I patched code and I have now panid and extpanid I request from config file. If I change panid in config and restart - I get another one. Fully correct behaviour. This is patch for herdsman 0.5.7 (one is installed currently with my dev branch). I wanted to make RP, but realised that in current dev of herdsman things are completely different.

panId2.patch.zip

The reason that this was done is that once a pan ID is already in used and one is specified, the coordinator does not start. But I agree that we should just pass this and let the user decide on a new pan ID.

ekaats commented 5 years ago

It does explain a the issues I've been having with 3.0. I haven't been able to keep a working network after reboots. I to did once succeed in getting the settings just right, but after reconfigurating things within home assistant and reflashing the stick my setup got unreliable as well.

Now I am running 1.2 with source routing and it seems to be stable although (re)pairing is somewhat challenging.

Aside from these issues, is there any idea on what the ideal future setup would be for zigbee 3.0 and the new herdsmen? Will CC2530/31 still be the main platform or are other options maturing enough for 30/31 to be considered obsolete? From the discussions I take that we are running against the limits of these sticks as coordinators. Is that correct?

choeflake commented 5 years ago

Aside from these issues, is there any idea on what the ideal future setup would be for zigbee 3.0 and the new herdsmen? Will CC2530/31 still be the main platform or are other options maturing enough for 30/31 to be considered obsolete? From the discussions I take that we are running against the limits of these sticks as coordinators. Is that correct?

That is also my concern, 40 devices on this stick is not enough. Even the 'new' (?) CC2652R has the limit of 40 devices. So what options do we have? Personally I like the way zigbee2mqtt is architectured and open source. Compared to the deCONZ stack for example.

Koenkk commented 5 years ago

We are definitely running into the limits of the CC2530/1, however the CC2652R should be able to support much more devices than 40, however currently I cannot get it to work stable (https://e2e.ti.com/support/wireless-connectivity/zigbee-and-thread/f/158/p/834603/3089128#3089128).

There are basically two options:

antst commented 5 years ago

You can also look at cc2538, I have coordinator firmware (3.0.2 stack) for it with limit of 200 directly connected devices. Not tested though at this scale. But 40-50 works like a charm.

https://github.com/Koenkk/zigbee2mqtt/issues/1568

imdos commented 5 years ago

@Koenkk Does this Q3 fix provide an improvement for CC253X(0/1/8) users as well? Or does this only have a benefit for the CC2652R sticks.

Koenkk commented 5 years ago

@imdos no, actually z-stack 3 on CC253X is already stable, I'm running it in my production setup for months now without issues.

imdos commented 5 years ago

Cool, well I guess it is time to switch then from 1.2 to 3. Will it be possible to upgrade and downgrade the hardware? So eg from 2531 to 2652r or 2538 and back if it turns out to be not so stable?

Koenkk commented 5 years ago

@imdos migration is not possible as the internal data structures of the firmware's are not compatible.

imdos commented 5 years ago

@imdos migration is not possible as the internal data structures of the firmware's are not compatible.

Oh; bummer, do I get this right? It won't be possible to switch the hardware without repairing (re-join each device)?

I understand it isn't possible to upgrade from 1.2 to 3 without repairing each device; but that is acceptable and should occur only once.

xppx99 commented 5 years ago

@imdos no, actually z-stack 3 on CC253X is already stable, I'm running it in my production setup for months now without issues.

@Koenkk , can you please confirm that you are using the 20190425 firmware version in your stable production tests please? Thanks

Koenkk commented 5 years ago

@xppx99 yes that's the one I'm using.

imdos commented 5 years ago

@imdos migration is not possible as the internal data structures of the firmware's are not compatible.

Oh; bummer, do I get this right, then? It won't be possible to switch hardware without repairing (re-join each device)?

I understand it isn't possible to upgrade from 1.2 to 3 without repairing each device; but that is acceptable and should thus occur only once.

electrofloat commented 5 years ago

@imdos https://github.com/Koenkk/zigbee2mqtt/issues/1445#issuecomment-536600472 https://github.com/Koenkk/zigbee2mqtt/issues/1445#issuecomment-537835729

Exact same comment twice, 3 days in between. What exactly are you trying to say here? You are asking a question and answering it at the same time.

peterforeman commented 5 years ago

I'm experiencing random connection problems with some lights with the latest zigbee2mqtt and 20190425 fw:

zigbee2mqtt:error 10/1/2019, 11:42:28 PM Zigbee publish to device 'xxx', genOnOff - off - {} - {"manufSpec":0,"disDefaultRsp":0} - null failed with error Error: AF data request fails, status code: 205. No network route. Please confirm that the device has (re)joined the network.

While this is a mains powered (Hue) device and worked before but suddenly doesn't anymore. I have at least two (powered) lights which randomly give these errors. The rest of the network is stable, better than it was with 1.2.

imdos commented 5 years ago

@imdos #1445 (comment) #1445 (comment)

Exact same comment twice, 3 days in between. What exactly are you trying to say here? You are asking a question and answering it at the same time.

I editted my first post in which my question wasn't clear (which was my first sentence.) My second sentence is actually already confirmed in another post.

Therefore I got the impression that the question was missed and had to validate this by asking the same again.

My apologies if that isn't accepted behaviour.

Koenkk commented 5 years ago

@peterforeman unfortunately Z-Stack 3 on the CC2531 consumes too much memory to enable the route caching features (that is why you are seeing this error). For this we need to use more powerful hardware (CC2652R/CC2538); both are in the process of being supported but still WIP.

imdos commented 5 years ago
  • Support non TI adapters, I've already did lots of preparations for this regarding the zigbee-herdsman refactor. Candidates are: XBEE S2c, nRF52840, Conbee 2, .....

@Koenkk Will this dongle be sufficient or do we require a much larger and more expensive board? There is no ipx nor SMA connector available, so I geuss less transmitting and receiving power.

antst commented 5 years ago

@Koenkk Will this dongle be sufficient or do we require a much larger and more expensive board? There is no ipx nor SMA connector available, so I geuss less transmitting and receiving power.

Don't rush! ) This is ideal candidate for coordinator, but it doesn't have even suitable firmware yet :)

vke commented 5 years ago

Don't rush! ) This is ideal candidate for coordinator, but it doesn't have even suitable firmware yet :)

Have you tried Nordic's CLI agent example?

antst commented 5 years ago

CLI exmaple isn't really what you need. It is full-featured coordinator but not full featured border-router (this is what we exactly use, but call "coordinator"). And text protocol isn't best option either. So, there is a need in firmware with binary protocol and fully-featured border-router functionality.

vke commented 5 years ago

CLI exmaple isn't really what you need.

I understand perfectly what you are talking about, as I am a developer familiar with nrf52840, but it seems to me that this is a good start point to implement something like ZNP-like proto on nrf52 series.

Koenkk commented 5 years ago

@vke @antst i think we should at least give the text protocol a try. The dongle is cheap; very fast and can be programmed over usb without the need of a flasher. @vke could you help with this?

antst commented 5 years ago

There is a whole thread on nordic forum about CLI not reporting out of the box attributes updates or whatever, I don't really recall what. But something which is required for coordinator. But, otherwise, what is there - it works, I checked :) And I played with NRF CLI as dedicated border-router (without coordinator function). It worked, but only with more or less "standard" zigbee devices (ie really written according to specs and standards of zigbee and ZCL).

sjorge commented 5 years ago

I did end up settling for CC2531 for now and stuck with the 1.2 firmware for now, but the nRF52840 USB dongles do look very appealing over a CC2652R cost wise. I still had one in my cart on amazon for ~ 15 EUR from when I first started looking, it went up to 18 EUR now but that is still < 20 EUR which is a very good price point.

Edit: picked up a nRF52840 I had in my card, as it is still super cheap and could be fun to mess around with.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

electrofloat commented 4 years ago

(activity for not closing it)

Kormyr commented 4 years ago

Starting the journey, thanks a lot @Koenkk.

So far just small pairing issues with the Aqara Opple buttons (need to try a lot of times to get it pairing), but otherwise it works fine 👍

Still waiting for some other Aqara devices to have the complete setup. IMG_20200117_073933

Find the 3 intruders that won't be using Zigbee2MQTT 😄

drbios commented 4 years ago

Starting the journey, thanks a lot @Koenkk.

So far just small pairing issues with the Aqara Opple buttons (need to try a lot of times to get it pairing), but otherwise it works fine

Still waiting for some other Aqara devices to have the complete setup. IMG_20200117_073933

Find the 3 intruders that won't be using Zigbee2MQTT

Hi How do you activate the external antenna in the coordinator?

Kormyr commented 4 years ago

Starting the journey, thanks a lot @Koenkk. So far just small pairing issues with the Aqara Opple buttons (need to try a lot of times to get it pairing), but otherwise it works fine Still waiting for some other Aqara devices to have the complete setup. IMG_20200117_073933 Find the 3 intruders that won't be using Zigbee2MQTT

Hi How do you activate the external antenna in the coordinator?

I did not :) I just had this antenna laying around that fits and wanted to show that it was the CC1352P-2. But I just noticed that I posted this message on the wrong topic...

matlab22 commented 4 years ago

Hi How do you activate the external antenna in the coordinator?

I did not :) I just had this antenna laying around that fits and wanted to show that it was the CC1352P-2. But I just noticed that I posted this message on the wrong topic...

To enable the external antenna you have to move a small resistor at the board. If you got your antenna from a wifi device it most commonly has a RP-SMA connector which does not fit the SMA on the board. You need an antenna with a SMA. Otherwise the middle part (small pin) will not connect properly.

Kormyr commented 4 years ago

Hi How do you activate the external antenna in the coordinator?

I did not :) I just had this antenna laying around that fits and wanted to show that it was the CC1352P-2. But I just noticed that I posted this message on the wrong topic...

To enable the external antenna you have to move a small resistor at the board. If you got your antenna from a wifi device it most commonly has a RP-SMA connector which does not fit the SMA on the board. You need an antenna with a SMA. Otherwise the middle part (small pin) will not connect properly.

Interesting, did anyone test to see if there is a real increase in signal range/quality with an external antenna?

I wanted to wait for somebody to try it to see if it's worth to add an antenna.

By the way the antenna on the picture is this one, which I assume should work: https://www.aliexpress.com/item/4000293397269.html

drbios commented 4 years ago

Hi How do you activate the external antenna in the coordinator?

I did not :) I just had this antenna laying around that fits and wanted to show that it was the CC1352P-2. But I just noticed that I posted this message on the wrong topic...

To enable the external antenna you have to move a small resistor at the board. If you got your antenna from a wifi device it most commonly has a RP-SMA connector which does not fit the SMA on the board. You need an antenna with a SMA. Otherwise the middle part (small pin) will not connect properly.

Interesting, did anyone test to see if there is a real increase in signal range/quality with an external antenna?

I wanted to wait for somebody to try it to see if it's worth to add an antenna.

By the way the antenna on the picture is this one, which I assume should work: https://www.aliexpress.com/item/4000293397269.html

LOL proud owner of 1352p-2 XD thanks for the antenna recomendation

matlab22 commented 4 years ago

Hi How do you activate the external antenna in the coordinator?

I did not :) I just had this antenna laying around that fits and wanted to show that it was the CC1352P-2. But I just noticed that I posted this message on the wrong topic...

To enable the external antenna you have to move a small resistor at the board. If you got your antenna from a wifi device it most commonly has a RP-SMA connector which does not fit the SMA on the board. You need an antenna with a SMA. Otherwise the middle part (small pin) will not connect properly.

Interesting, did anyone test to see if there is a real increase in signal range/quality with an external antenna?

I wanted to wait for somebody to try it to see if it's worth to add an antenna.

By the way the antenna on the picture is this one, which I assume should work: https://www.aliexpress.com/item/4000293397269.html

The pictures of the antenna looks good, however the description is confusing. So if the antenna has this little middle pin, then you are probably fine :) I did make a brief test with PCB antenna and external antenna and found no significant change. So I put the resistor back and stayed with the quite good PCB antenna

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

electrofloat commented 4 years ago

Koenkk just made huge progress towards Z-Stack3 Green Power here: https://github.com/Koenkk/zigbee-herdsman/issues/22#issuecomment-610544376

jdtsmith commented 4 years ago

Just tried updating my CC2531 with 3.0 stack to hopefully fix IKEA repeater and blinds dropping off network. I used the firmware CC2531ZNP-with-SBL.hex from CC2531_20190425.zip. A fresh dev install of zigbee2mqtt and a vanilla configuration.yaml fails to start with the following error: 

 Starting zigbee-herdsman...
Apr 12 21:58:15 screenpi npm[26975]: zigbee2mqtt:error 2020-04-12 21:58:15: Error while starting zigbee-herdsman
Apr 12 21:58:15 screenpi npm[26975]: zigbee2mqtt:error 2020-04-12 21:58:15: Failed to start zigbee
Apr 12 21:58:15 screenpi npm[26975]: zigbee2mqtt:error 2020-04-12 21:58:15: Exiting...
Apr 12 21:58:15 screenpi npm[26975]: zigbee2mqtt:error 2020-04-12 21:58:15: Error: SREQ '--> SYS - osalNvRead - {"id":96,"offset":0,"len":1,"value":{"type":"Buffer","data":[85]}}' failed with status '2' (expected '0')
Apr 12 21:58:15 screenpi npm[26975]:     at Znp.<anonymous> (/home/pi/zigbee2mqtt/node_modules/zigbee-herdsman/dist/adapter/z-stack/znp/znp.js:246:27)
Apr 12 21:58:15 screenpi npm[26975]:     at Generator.next (<anonymous>)
Apr 12 21:58:15 screenpi npm[26975]:     at fulfilled (/home/pi/zigbee2mqtt/node_modules/zigbee-herdsman/dist/adapter/z-stack/znp/znp.js:5:58)

Tried re-flashing firmware with same results.

papanirual commented 4 years ago

I just updated to the firmware from the version of 2020/03/06 (which worked fine) to the one of 2020/03/28 and also updated zigbee2mqtt to dev version ("transportrev":2,"product":1,"majorrel":2,"minorrel":7,"maintrel":1,"revision":20200306}) and am now also stuck with the same error as @jdtsmith.

Going back to the previous firmware did not help, also reflashing it with the steps from https://www.zigbee2mqtt.io/information/flashing_via_uniflash.html did not fix the problem.

[Update 19:26]: I went back to the newest zigbee master version, now the system is running again.