Koenkk / zigbee2mqtt

Zigbee 🐝 to MQTT bridge 🌉, get rid of your proprietary Zigbee bridges 🔨
https://www.zigbee2mqtt.io
GNU General Public License v3.0
11.77k stars 1.64k forks source link

Sonoff Zigbee3.0 USB Dongle pairing issues #9117

Closed notenoughtech closed 2 years ago

notenoughtech commented 2 years ago

I got serious pairing issues with various Zigbee 3.0 end devices. Sonoff Zigbee3.0 USB Dongle was using {"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20210120,"transportrev":2},"type":"zStack3x0"} and the latest firmware after a successful flash. With the latest firmware, I was not able to get anything to pair - no MQTT messages about the interview process. So I reverted back to the 20210120 revision and I was able to pair Aqara TVOC and IKEA button, but Sonoff devices or Xiaomi buttons were not even entering the interview stage. The same devices were tested with ZZH and CC2531 and I was able to pair and use the devices.

There isn't much information I can share, as the log wasn't being populated. Happy to try things out

Koenkk commented 2 years ago

Could you flash this FW: https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/CC1352P2_CC2652P_launchpad_coordinator_20210708.zip and use a CC2531 to sniff the traffic when pairing fails? https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html

notenoughtech commented 2 years ago

I will give it a go this eve.

notenoughtech commented 2 years ago

On this firmware I see pairing messages -- so there is progress

Some observations: AqaraE1: Paired without even triggering the pairing procedure (was it waiting to pair for 2 days?) Sonoff button: - no dice not responsive: Oct 11 19:16:13 testpi npm[2173]: Zigbee2MQTT:error 2021-10-11 19:16:13: Failed to configure '0x00124b001cd61409', attempt 3 (Error: Bind 0x00124b001cd61409/1 genOnOff from '0x00124b0022a54d11/1' failed (AREQ - ZDO - bindRsp after 10000ms) and also triggered Oct 11 19:11:02 testpi npm[2173]: (node:2188) UnhandledPromiseRejectionWarning: AssertionError [ERR_ASSERTION]: No image available for imageType '14976' Oct 11 19:11:02 testpi npm[2173]: at getImageMeta (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/zigbeeOTA.js:24:5) Oct 11 19:11:02 testpi npm[2173]: at processTicksAndRejections (internal/process/task_queues.js:95:5) Oct 11 19:11:02 testpi npm[2173]: at isNewImageAvailable (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/common.js:265:18) Oct 11 19:11:02 testpi npm[2173]: at Object.isUpdateAvailable (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/lib/ota/common.js:256:23) Oct 11 19:11:02 testpi npm[2173]: at OTAUpdate.onZigbeeEvent (/opt/zigbee2mqtt/lib/extension/otaUpdate.ts:61:31)

Sonoff Motion sensor - at first I thought it would fail but took 2 min to pair and works ok now 🤷‍♂️

Lidl stuff, before not able to pair but worked just like that.

Sonoff temp sensor: gets stuck on Oct 11 19:36:43 testpi npm[2173]: Zigbee2MQTT:info 2021-10-11 19:36:43: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_successful","meta":{"description":"Temperature and humidity sensor","friendly_name":"0x00124b002269c2c8","model":"SNZB-02","supported":true,"vendor":"SONOFF"},"type":"pairing"}'

gave up after 5 min

Some improvement there over what it was before - I wasn't on the launchpad version. I need to setup sniffing to give you more sets, got lots on my plate this week so I will try to post as soon as I can

Koenkk commented 2 years ago
notenoughtech commented 2 years ago

Any suggestions for big delays in pairing? I noticed stopping system also took some time with this stick.

On Tue, 12 Oct 2021, 07:23 Koen Kanters, @.***> wrote:

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-940702457, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL23NELWT7S46FHIKPADUGPH7XANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Koenkk commented 2 years ago

If you provide a debug log + sniff starting from the moment you put the device into pairing mode I can check that.

io53 commented 2 years ago

@notenoughtech can you make sure you are not running into this linux kernel driver bug? https://github.com/Koenkk/zigbee2mqtt/issues/8663 Probably not if zzh works as it should, but worth a look.

notenoughtech commented 2 years ago

This sound very familiar if I'm honest

On Tue, 12 Oct 2021, 08:04 Elias Berg, @.***> wrote:

@notenoughtech https://github.com/notenoughtech can you make sure you are not running into this linux kernel driver bug? #8663 https://github.com/Koenkk/zigbee2mqtt/issues/8663 Probably not if zzh works as it should, but worth a look.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-940724670, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL2ZSZLXE3PYHR4DT4FDUGPMYHANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Hedda commented 2 years ago

@notenoughtech can you make sure you are not running into this linux kernel driver bug? #8663

He should not run into the bug mentioned in https://github.com/Koenkk/zigbee2mqtt/issues/8663 if his issue is with the new "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P")" as it uses Silicon Labs CP2102N (from Silabs CP210x Family) USB-to-UART bridge chip instead of WCH CH341.

@notenoughtech you should maybe clarify in your original post that your issue here is specifically with "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P" based on Texas Instruments CC2652P)" and not with their previous "Zigbee 3.0 USB Dongle (SKU: "9888010100045" based on Silicon Labs EFR32MG21)", as both are from ITead but only the "Zigbee 3.0 USB Dongle (SKU: "9888010100045") uses a CH341 USB-to-UART bridge chip.

At least that is my assumption considering the timing of this and your recent review article:

https://notenoughtech.com/home-automation/sonoff-zigbee-3-0-usb-dongle-plus/

https://www.reddit.com/r/Not_Enough_Tech/comments/q2vfq4/getting_started_with_sonoff_zigbee_30_usb_dongle/

notenoughtech commented 2 years ago

Since the dongle plus comes with an enclosure, I think viewers will figure out which I'm talking about (I mentioned in the article/video the zigbee IC model the PCB hosts.

Mat Zolnierczyk Owner: NotEnoughTECH Editor: *pocketables.com http://www.pocketables.com/about, xda-developers.com https://www.xda-developers.com/author/notenoughtech/ E: @. W: www.notenoughtech.com https://inbox.google.com/u/1/www.notenoughtech.com | Twitter: @notenoughtech https://twitter.com/NotEnoughTECH

On Wed, Oct 27, 2021 at 2:32 PM Hedda @.***> wrote:

@notenoughtech https://github.com/notenoughtech can you make sure you are not running into this linux kernel driver bug? #8663 https://github.com/Koenkk/zigbee2mqtt/issues/8663

He should not run into that bug if the issue is with "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P") https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/" as that uses Silicon Labs CP2102N https://www.silabs.com/interface/usb-bridges/usbxpress/device.cp2102n-gqfn20 (from Silabs CP210x Family) USB-to-UART bridge chip instead of the WCH CH341.

@notenoughtech https://github.com/notenoughtech you should clarify that your issue here is specifically with "SONOFF Zigbee 3.0 USB Dongle Plus (Model: "ZBDongle-P" based on Texas Instruments CC2652P) https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/" and not with their previous "Zigbee 3.0 USB Dongle (SKU: "9888010100045" based on Silicon Labs EFR32MG21) https://itead.cc/product/zigbee-3-0-usb-dongle/", as both are from ITead but only the "Zigbee 3.0 USB Dongle (SKU: "9888010100045") uses a CH341 USB-to-UART bridge chip.

At least that is my assumption considering the timing of this and your recent review article:

https://notenoughtech.com/home-automation/sonoff-zigbee-3-0-usb-dongle-plus/

https://www.reddit.com/r/Not_Enough_Tech/comments/q2vfq4/getting_started_with_sonoff_zigbee_30_usb_dongle/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-952934004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL27VHSX7OQYU22R4YE3UJAEPTANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Hedda commented 2 years ago

FYI, perhaps not quite general knowledge but other than the obvious in avoiding having your Zigbee coordinator close to your Wi-Fi router/access-points, know that you can often get Zigbee pairing issues when got electromagnetic interference from USB 3.0 devices and cables connected to those devices are known to often interfere with Zigbee coordinator signal reception:

https://www.intel.com/content/www/us/en/products/docs/io/universal-serial-bus/usb3-frequency-interference-paper.html

Simplest solution to avoid EMF from devices is to use a long (1-2 meter) USB-extension-cable to get your ZB dongle a bit away.

Maybe little of topic, but FYI, I have collected these general best practices for avoiding pairing difficulties with Zigbee devices:

Best practices for avoiding pairing difficulties

Also FYI, while on the subject of your issues I have also connected these general tips on improving Zigbee network range:

Tips on improving Zigbee network range

Both poor signal quality or signal interference can lead to transmission errors and related issues. This section contains fundamental troubleshooting tips on how to improve signal quality plus optimize range and coverage. Improving signal quality and removing sources of signal interference can in most cases maximize range and resolves problems related to transmission errors. Please try to follow at least some of these recommendations before posting on the community forums or reporting issues to developers.

  1. Adding more Zigbee routers between devices far away and the next closest router or your Zigbee coordinator. Zigbee network topology uses a "mesh network" design which means that each device that acts as a Zigbee router will extend the total range and coverage of your Zigbee network as a whole. The solution is to distribute more Zigbee routers in areas with poor reception. Note that while there are exceptions, understand that almost all permanently powered devices will serve as a Zigbee router. Thus adding more permanently powered Zigbee devices will allow greater range better coverage. There are also dedicated Zigbee routers which you can find by doing a community search for "Zigbee signal repeater" or "Zigbee range extender") (an example being IKEA Tradfri Signal Repeater). Devices that can not act as routers are typically battery-operated and known as "end devices". There are some exceptions as some end devices (e.g. Xiaomi/Aqara devices) have issues connecting through routers that are not from the same manufacturer.
  2. Use a long USB extension cable with your Zigbee coordinator adapter. This not only enables you to position the Zigbee coordinator adapter for better signal quality, but it also allows you to locate the Zigbee coordinator further away from Wi-Fi access-points/routers or other sources of 2.4GHz signals to avoid signal interference. Optimally in the best area in your home depending on your building's floorplan. Ideally, you want to place the Zigbee coordinator somewhere in the middle of your house/apartment. Building materials influence signal quality, for example, dense/thick concrete, bricks, masonry, etc. dampen all wireless signals. Place the Zigbee coordinator with some distance from walls, ceilings and floors. Also, try different orientations of your Zigbee antenna (or your whole Zigbee coordinator adapter if it has an internal antenna). Some Antennas have a stronger signal in a certain direction. Simply changing orientation can improve signal quality already. Note: A bad USB extension cable may lead to connection issues between the system and the Zigbee coordinator adapter, symptoms of this are disconnection messages.
  3. Zigbee and Wi-Fi can operate on various channels in the 2.4GHz spectrum. A busy Wi-Fi access-points/routers that are operating in the same frequency range (overlapping channels) as your Zigbee coordinator will drown out the Zigbee traffic, especially if they are located close to each other. To avoid interference between Zigbee and Wi-Fi try to choose channels without overlap. Check the channel your Wi-Fi access-points/routers are using (either by checking on the router's web interface or using a Wi-Fi analyzer app). Changing the channel of the Zigbee network usually requires reforming it and re-join/re-pair all of your Zigbee devices. It is therefor typically it's a lot easier to just change the channel used by your Wi-Fi router and/or access-points. See this article for Wi-Fi and Zigbee channels coexistance to avoid using overlapping frequency ranges.
  4. Check if updating the firmware on your Zigbee coordinator adapter and your Zigbee end devices is possible. Note that depending on your hardware the latest Zigbee coordinator firmware might not always be the one recommended by the community so it is advised to ask before upgrading.
  5. If your Zigbee coordinator adapter has a removable antenna (e.g., with an SMA-connector) then you have the option of replacing it with a high-gain antenna. Note that antennas with higher gain usually have directionality: You might have better reception on the same floor, but reception across floors might suffer. In addition, you also have the option to use an antenna extension cable if needed (usually using just a USB extension cable for your Zigbee coordinator adapter is the better alternative). This should really only be needed if you are trying to cover a long distance, like to another building or very dense/thick walls, ceilings and floors.
  6. Buy more powerful Zigbee radio hardware with a better radio range, preferably with an external antenna and based on a newer chip that offers up-to-date firmware. If you are not only experimenting with Zigbee but want a permanently stable and healthy Zigbee mesh network with potentially many devices then you should consider upgrading to a more powerful and newer Zigbee coordinator USB adapter or Ethernet to serial gateway/bridge. Generally, Zigbee adapters with an external antenna will have a better range and offer you more flexibility, therefore you will also want to avoid buying an internal Zigbee adapter unless it has a port for an external antenna.
Hedda commented 2 years ago

@notenoughtech btw, regarding your range tests; Koenkk now stated current firmware doesn't yet support +20 dbm output:

https://github.com/Koenkk/zigbee2mqtt/discussions/8840#discussioncomment-1550449

That means that this CC2652P based Sonoff Zigbee 3.0 USB Dongle Plus adapter with its current firmware version only operate at +5 dBm output, so no wonder it does not perform better when compared to Electrorama’s zzh dongle which is based on CC2652R that physically only capable of +20 dBm output. Thus only once we get a new community firmware with RF switch configured will enabling +20 dBm output in application settings actually work.chiakikato also manually tested Sonoff Plus dongle hardware and confirms it support +20 dBm output:

https://github.com/Koenkk/zigbee2mqtt/discussions/8840#discussioncomment-1550148

Guess this means you have to redo the tests and update review when get new firmware.

Koenkk commented 2 years ago

@Hedda before people get confused,

CC2652R that physically only capable of +20 dBm output

This should be 5dbm, not 20dbm, so theoretically the ZZH should gave the same performance as the sonoff adapter currently.

notenoughtech commented 2 years ago

I will be keen to redo this (time is a massive issue right now) at some point as it would make a good article

On Fri, 29 Oct 2021, 15:41 Koen Kanters, @.***> wrote:

@Hedda https://github.com/Hedda before people get confused,

CC2652R that physically only capable of +20 dBm output

This should be 5dbm, not 20dbm

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-954799456, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL22VCWC4YJLPJRWZYHTUJK6A3ANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Koenkk commented 2 years ago

@notenoughtech do you still have the pairing issues? If so could you provide me the required debug information so I can investigate this further? https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-940713642

notenoughtech commented 2 years ago

Other than initial feedback I didn't have time to play more with it. I need to sit down and get the proper sniffing log from cc2531 as you requested before.

My social calendar, work and NotEnoughTECH is a bit stretched 😭😭

On Fri, 29 Oct 2021, 16:08 Koen Kanters, @.***> wrote:

@notenoughtech https://github.com/notenoughtech do you still have the pairing issues? If so could you provide me the required debug information so I can investigate this further? #9117 (comment) https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-940713642

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-954819772, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL2ZOEOTKG7CEFV3LRX3UJLBGFANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

hoggerz commented 2 years ago

Hi Koen,

I’ve recently purchased a couple of these, straight away with the latest 3.X.0 branch firmware, and also with the newer one you posted specifically referencing the sonoff dongle, I found issues with pairing Ikea bulbs. Oddly some that had older firmware versions paired first time with no issues, but newer firmware bulbs always had issues either pairing or setting up reporting for the bulbs. I had similar issues with some very old HUE b22 bulbs problems I will sniff the traffic of various items when I’m back home at the weekend.

hoggerz

u8915055 commented 2 years ago

Id like to leave a comment here as well. I purchased a couple of the sonoff zigbee 3.0 plus dongles and have not been able to get them to work. Now im new to zigbee2mqtt.. ive been a long time ZHA user through HA but want to move to z2m to get better device support.

I am running this on a debian vm, fresh install. I followed all the instructions to install z2m. I took my dongle and flashed it with 20210708. I was able to get z2m started and the web interface came up etc. It says its listening for devices to join but i cant get a single device to join. No logs no nothing (even with debug logging enabled) It just sits there. I have flashed this dongle on both a windows vm and also using the CC2538-bsl script.. both seemed to flash and verify successfully but still wont enable any devices to join.

I have about 25 zigbee devices that are all running fine with ZHA. I am running on channel 20.

I have found though that ive run into that problem with starting z2m raises that duplicate pan_id issue.. I was able to get around that by re-flashing and setting the pan_id to GENERATE in teh config file. That sometimes works. What always seems to let me get things started is to remove the external antenna. But that fact sort of suggests to me that something is happening at the stack level because with the antenna installed i hit the pan_id issue and without z2m will start.

Anyway i know im not giving a lot of cold hard facts here but that's where things stand for me. Unable to get a single device to join. Im thinking of flashing a CC2531 device and see if i can get that to accept devices on the same setup.

Happy to test or try anything. Thanks in advance.

u8915055 commented 2 years ago

Wanted to add some more info here.. i took my other sonoff 3.0 dongle from its package, disconnected the antenna.. replaced the flashed one, and restarted zigbee2mqtt, and then reattached the antenna. I am now able to get a few devices to connect.. it says in the web interface that its running firmware 20210708 as well.. strange why this one works and the other one doesnt. One may say i flashed it wrong but i did it a few times and did full verification on the written image and it checked out. Im going to take the original one and use it with ZHA and see if it works fine there.

Now im having other issues iwth it reading battery levels, not sure whats happening there, but at least its reading some devices.

Hedda commented 2 years ago

Wanted to add some more info here.. i took my other sonoff 3.0 dongle from its package, disconnected the antenna.. replaced the flashed one, and restarted zigbee2mqtt, and then reattached the antenna. I am now able to get a few devices to connect.. it says in the web interface that its running firmware 20210708 as well.. strange why this one works and the other one doesnt. One may say i flashed it wrong but i did it a few times and did full verification on the written image and it checked out. Im going to take the original one and use it with ZHA and see if it works fine there.

Sounds like a hardware fault with the other one of your adapters if one adapter is working and the other adapter is not when both are flashed with same firmware image. Maybe try switching antenna between them to rule out one of the antennas being at fault.

FYI, once fixed know there is more general and broader discussion about ITead's Sonoff Zigbee 3.0 USB Dongle Plus here -> https://github.com/Koenkk/zigbee2mqtt/discussions/8840

Nobody931 commented 2 years ago

Hello, I ordered the Sonoff Zigbee3.0 USB Dongle (ITEAD) two weeks ago and it arrived yesterday. Before I used the CC2530 with serial connection the my raspberry Pi4 which worked good, however, after updating the raspberry pi I was not able to connect the CC2530, so I order the Sonoff Dongle (as I read that it does not perform that good anymore). I was able to connect the Dongle and pair with a xiaomi motion detector (with battery status). However, I am not able to pair my other devices which compromises of IKEA light bulbs, e1743 buttons and other supported devices. I found this issue and before I try reflashing the USB Dongle (which already shipped with the latest firmware [20210708]) I thought maybe there is another problem and used the cc2531 sniffer for capturing the communication. This is the capture of the e1743 button with a fresh battery. logAndCapture.zip I hope the log and capture is good enough, if necessary I can provide more log and capture other devices as well if time allows it :) Would love the get my old setup working, thank you! tldr: Changed from CC2530 to pre-flashed Sonoff Dongle, successfully paired motion sensor, but not able to pair other (supported and functioning) devices.

u8915055 commented 2 years ago

 Yes I found the sonoff dongle just doesn’t work as well as my old setup with a sonoff zhabridge with tasmota. I really think zigbee2mqtt looks really interesting but I can’t get it working with this adapter. I just ordered another adapter, from here. https://slae.sh/projects/cc2652/#get-one I’m going to try it with this. Hopefully I get a better outcome.

For the record I have about 25 devices and with the sonoff 3.0 stick I tried about six of them. Four came in fine and two would not pick up the battery state. I tried many times to re pair them. No luck. All these devices worked fine with ZHa.

On Dec 4, 2021, at 9:34 AM, Nobody931 @.***> wrote:  Hello, I ordered the Sonoff Zigbee3.0 USB Dongle (ITEAD) two weeks ago and it arrived yesterday. Before I used the CC2530 with serial connection the my raspberry Pi4 which worked good, however, after updating the raspberry pi I was not able to connect the CC2530, so I order the Sonoff Dongle (as I read that it does not perform that good anymore). I was able to connect the Dongle and pair with a xiaomi motion detector. However, I am not able to pair my other devices which compromises of IKEA light bulbs, e1743 buttons and other supported devices. I found this issue and before I try reflashing the USB Dongle (which already shipped with the latest firmware [20210708]) I thought maybe there is another problem and used the cc2531 sniffer for capturing the communication. This is the capture of the e1743 button with a fresh battery. logAndCapture.zip I hope the log and capture is good enough, I necessary I can provide more log and capture other devices as well if time allows it :) Would love the get my old setup working, thank you! tldr: Changed from CC2530 to pre-flashed Sonoff Dongle, successfully paired motion sensor, but not able to pair other (supported and functioning) devices.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

flatsiedatsie commented 2 years ago

My Sonoff 3.0 Plus USB stick arrived. I've been trying it on a fresh Linux device (Webthings Gateway), with a brand new random PAN_ID and Network_Key.

I tried to switch an IKEA light over from my main system (with Conbee2), to this new test system.

So far I have not been able to get it to pair.

I have tried:

At some point I noticed an SRSP - ZDO - mgmtPermitJoinReq after 6000mserror:

2021-12-20 22:19:19.703 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"value": true}'
2021-12-20 22:19:19.705 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.707 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:19: Zigbee: allowing new devices to join.
2021-12-20 22:19:19.709 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.712 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/config/devices/get' with data ''
2021-12-20 22:19:19.713 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.714 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:error 2021-12-20 22:19:25: Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
2021-12-20 22:19:25.717 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.772 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:25: Error: SRSP - ZDO - mgmtPermitJoinReq after 6000ms
2021-12-20 22:19:25.774 INFO   : zigbee2mqtt-adapter:     at Timeout.waiter.timer.setTimeout [as _onTimeout] (/home/pi/.webthings/data/zigbee2mqtt-adapter/zigbee2mqtt/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35)
2021-12-20 22:19:25.776 INFO   : zigbee2mqtt-adapter:     at ontimeout (timers.js:436:11)
2021-12-20 22:19:25.778 INFO   : zigbee2mqtt-adapter:     at tryOnTimeout (timers.js:300:5)
2021-12-20 22:19:25.779 INFO   : zigbee2mqtt-adapter:     at listOnTimeout (timers.js:263:5)
2021-12-20 22:19:25.781 INFO   : zigbee2mqtt-adapter:     at Timer.processTimers (timers.js:223:10)
2021-12-20 22:19:25.782 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.784 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:25: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error"}'

But most of the time there's simply... nothing.

I still have to try:

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

Koenkk commented 2 years ago

@flatsiedatsie make sure to check https://www.zigbee2mqtt.io/guide/faq/#zigbee2mqtt-crashes-after-some-time

dreesi commented 2 years ago

My Sonoff 3.0 Plus USB stick arrived. I've been trying it on a fresh Linux device (Webthings Gateway), with a brand new random PAN_ID and Network_Key.

I tried to switch an IKEA light over from my main system (with Conbee2), to this new test system.

So far I have not been able to get it to pair.

I have tried:

  • powering off the main controller with the Conbee2 stick, just to make sure it wasn't interfering.
  • moving the Sonoff dongle and lightbulb closer together
  • many reboots and full power-off situations.

At some point I noticed an SRSP - ZDO - mgmtPermitJoinReq after 6000mserror:

2021-12-20 22:19:19.703 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"value": true}'
2021-12-20 22:19:19.705 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.707 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:19: Zigbee: allowing new devices to join.
2021-12-20 22:19:19.709 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:19.712 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:19: Received MQTT message on 'zigbee2mqtt/bridge/config/devices/get' with data ''
2021-12-20 22:19:19.713 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.714 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:error 2021-12-20 22:19:25: Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
2021-12-20 22:19:25.717 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.772 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:debug 2021-12-20 22:19:25: Error: SRSP - ZDO - mgmtPermitJoinReq after 6000ms
2021-12-20 22:19:25.774 INFO   : zigbee2mqtt-adapter:     at Timeout.waiter.timer.setTimeout [as _onTimeout] (/home/pi/.webthings/data/zigbee2mqtt-adapter/zigbee2mqtt/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35)
2021-12-20 22:19:25.776 INFO   : zigbee2mqtt-adapter:     at ontimeout (timers.js:436:11)
2021-12-20 22:19:25.778 INFO   : zigbee2mqtt-adapter:     at tryOnTimeout (timers.js:300:5)
2021-12-20 22:19:25.779 INFO   : zigbee2mqtt-adapter:     at listOnTimeout (timers.js:263:5)
2021-12-20 22:19:25.781 INFO   : zigbee2mqtt-adapter:     at Timer.processTimers (timers.js:223:10)
2021-12-20 22:19:25.782 INFO   : zigbee2mqtt-adapter: 
2021-12-20 22:19:25.784 INFO   : zigbee2mqtt-adapter: z2m stdout: Zigbee2MQTT:info  2021-12-20 22:19:25: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error"}'

But most of the time there's simply... nothing.

I still have to try:

  • extension cable
  • pairing different devices

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

same for me, ordered the sonoff stick but IKEA bulbs aren't pairing. I didn't start debugging yet and will try to follow the points Koenkk advised.

Hedda commented 2 years ago

I don't want to flash a different firmware, as for my purposes I'm looking for a cheap USB stick that I can recommend others to buy. It needs to be globally available, and work out of the box.

I highly recommend that you at least change the requirement to have a relatively fast/easy way for users to upgrade the Zigbee coordinator firmware on whatever adapter you choose. Suggest that you check out this "feature/ITead_Sonoff_Zigbee-delay" branch from JelmerT patch for cc2538-bsl which makes upgrading firmware much easier than following ITead's official guide (PR https://github.com/JelmerT/cc2538-bsl/pull/114 with patch for this sonoff plus dongle) -> https://github.com/JelmerT/cc2538-bsl/tree/feature/ITead_Sonoff_Zigbee-delay

Would therefore strongly suggest that you slightly change your particular "works out-of-the-box" condition requirement for fast/easy when looking at any Zigbee Coordinator adapter as upgrading the firmware at some point will always be a minimum requirement if you want a stable Zigbee network. If not direct when you bought it (because you got lucky and received stock when the manufacturer just happen to have updated to a new revision) then later on once you discover bugs on there is a security exploit.

Remember that all hardware today will need updates for firmware/drivers/software, so understand that by today's standard suggesting that users should never ever have to update their Zigbee coordinator firmware is almost like suggesting that users should be able to buy a smartphone and never update it either, (then thinking they can ask for support and not expecting being told to upgrade before getting any help).

Regardless of the standard dilemma for Zigbee DIY solutions is that you can today not really find any Zigbee coordinator adapter hardware and Zigbee software solution that is fast/easy, cheap, and good: You can only pick two of those.

image

flatsiedatsie commented 2 years ago

ON TOPIC:

I just tried flashing the latest development firmware (CC1352P2_CC2652P_launchpad_coordinator_20211217) onto the Sonoff stick, using the excellent Python script recommended by @Hedda

I still cannot pair my IKEA lightbulb.

I still don't see any reaction from Zigbee2MQTT.

OFF TOPIC:

highly recommend that you at least change the requirement to have a relatively fast/easy way to upgrade the Zigbee coordinator firmware on whatever adapter you choose.

Yes that's definitely something I'd like too. I'd like to build firmware updating of the USB stick into the Webthings Addon I work on. I've just updated that addon to add security support. I could add update flashing next.

feature/ITead_Sonoff_Zigbee-delay

Very nice. If I understand correctly, USB sticks like Zig-Ah-Zig-Ah also have this feature.

This is my list of desired features for a USB Zigbee stick:

Nice to have:

So far I've been recommending the Conbee2 stick. I've never once upgraded it since purchasing it years ago, but it seems to work fine so far.Theoretically it can be updated from the linux command line (which I'm attempting now, but I'm not having much luck so far)

The mayor downsides of the Conbee2 are:

TLDR: if the Sonoff stick could be made to work well, I think it's a winner.

Hedda commented 2 years ago

feature/ITead_Sonoff_Zigbee-delay

Very nice. If I understand correctly, USB sticks like Zig-Ah-Zig-Ah also have this feature.

@flatsiedatsie Yes such "Auto BSL" (formerly known as BootStrap Loader) feature allows you to trigger entering into bootloader mode via software and is built into many new microcontrollers, including CC2538, CC2652 and CC1352, but not supported by all:

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

Auto BSL is however not a feature of CC2530 and CC2531, however for those you can manually force it to enter bootloader and then flash them with zigpy-znp tools -> https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md#flash-operations but on the other hand CC2530 and CC2531 are no longer recommended for Zigbee2MQTT at all, especially not s a new purchase for new installations by beginners and users new to Zigbee.

I'd like to build firmware updating of the USB stick into the Webthings Addon I work on. I've just updated that addon to add security support. I could add update flashing next.

Off-topic; Then suggest you check out and take inspiration from ESP Web Tools https://esphome.github.io/esp-web-tools/ (which support OTW flashing of firmware via a web browser using Web Serial API, though currently only for ESP based MCU chips), as well a https://github.com/electrolama/llama-bsl (fork of cc2538-bs with some added auto-detection feature and more similar features planned to make OTW flashing of firmware more user-friendly)

This is my list of desired features for a USB Zigbee stick:

  • Works with a wide range of devices, including IKEA buttons and sensors.
  • Pairing is usually succesfull on the first try.
  • Can handle enough devices for an average home and has enough range by itself.
  • Can easily be purchased all over the world and at any time through 'big' retail channels.
  • Likely comes with a working firmware at purchase time.
  • Can be upgraded through Linux command.

You need to remember that most of the things on your list do not depend on the Zigbee hardware adapter or even the Zigbee stack firmware, instead, those things will rather depend on the Zigbee software solution and the Zigbee devices that you connect. Understand that Zigbee is the wild wild west as not all manufacturers do not strictly follow the rules, and all of the rules are not even strict rules but more like guidelines. So implementing support for different Zigbee devices from hundreds or thousands of manufacturers is hard and complex. Jeedom developers recently wrote a great blog post article which makes for a good read summing all that up better than I can:

https://blog.jeedom.com/6107-parlons-zigbee/

A couple of quick tips for any new Zigbee user are; upgrade Zigbee coordinator firmware before beginning. Stay away from Aqara, Xiaomi, as well as Tuya devices. When starting to add devices begin by adding at least a few main-powered devices that will work as Zigbee router (a.k.a. repeater) devices, preferably flash CC2652 and/or CC1352 adapters with Zigbee router firmware and use those as known working Zigbee routers to create a stable network mesh at its core.

The mayor downsides of the Conbee2 are:

  • The Zigbee2MQTT project doesn't seem to reallty recommend it?
  • It's relatively expensive.

For now, recommend using Texas Instruments based adapters with Zigbee2MQTT. ConBee/RaspBee support is only experimental is Zigbee2MQTT. So is Silicon Labs (Silabs EZSP) based adapters.

https://www.zigbee2mqtt.io/information/supported_adapters.html

Texas Instruments Z-Stack 3 based asapters also work with Home Assistant's ZHA and Jeedom's Zigbee plugin

ConBee/RaspBee is currently really only good if used with dresden elektronik's own deCONZ software and not third-party.

flatsiedatsie commented 2 years ago

Update: success! I was just able to pair an IKEA bulb with the Sonoff 3.0 Plus stick running the latest development coordinator firmware. It even paired on the first attempt this time.

Turns out there was an error in my code, and because of that the security wasn't being implemented. This meant that the new Raspberry Pi was still using the default PAN_ID and Network_key.

I suspect that the IKEA bulb didn't see the need to pair with the same controller it was technically already paired too.

nicoolaro commented 2 years ago

Update: success! I was just able to pair an IKEA bulb with the Sonoff 3.0 Plus stick running the latest development coordinator firmware. It even paired on the first attempt this time.

Turns out there was an error in my code, and because of that the security wasn't being implemented. This meant that the new Raspberry Pi was still using the default PAN_ID and Network_key.

I suspect that the IKEA bulb didn't see the need to pair with the same controller it was technically already paired too.

care to explain details i am having same problem with Ikea

flatsiedatsie commented 2 years ago

@nicoolaro I'm not sure what to say. I think it all worked as normal after I flashed the beta firmware to the Sonoff stick. In my case I had two conflicting controllers with the same PAN_ID, which I believe is what was causing my issues.

Hedda commented 2 years ago

Update! Z-Stack 3.x.0 20211217 firmware release image has now been moved to Koenkk's master and should be seen as stable:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.x.0/bin

Again, recommend upgrading firmware using JelmerT 's "ITead_Sonoff_Zigbee-delay" branch from this pull request

https://github.com/JelmerT/cc2538-bsl/pull/114

Hopefully ITead will begin pre-flashing this new release to future batches, just newer SimpleLink SDK contain many bug-fixes:

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/CHANGELOG.md

However note that keep upgrading to newer bug-fix releases will still always be recommended.

flatsiedatsie commented 2 years ago

I bought another Sonoff dongle to test.

Out of the box it did connect to IKEA devices this time. This confirms that my previous issues were causes by having two clashing PAN ID's.

TeslaCLT commented 2 years ago

I also have just receive the Sonoff Zigbee 3.0 USB Dongle Plus from ITead. It is running the zStack3x0 firmware at level 20210708. As others here saw, I have had tons of pairing problems. This is a fresh install, so no existing data being migrated over (I'm coming off SmartThings).

My first Ikea bulb paired right away without issue, but after none of my Ikeas, Sengleds, or eWeLights pair normal and fail during the interview. However, if I shut down z2m and run herdsman in debug mode via "DEBUG=zigbee-herdsman* npm start", they all pair without issue on the first try (from my perspective, one device did have to have multiple attempts per the log).

I'm not comfortable yet with flashing to another firmware, but as I get better acquainted with the coordinator, I'll give it a shot. I just wanted to share how debug mode helped me in case it helps other not rip their hair out!

flatsiedatsie commented 2 years ago

I just did another test.

Before I disabled debugging I was able to pair an IKEA motion sensor.

Them I tried to pair a bulb (TRADFRI LED bulb E26/E27 806 lumen, dimmable, warm white ), and it paired normally. I can toggle it and change its brightness.

I then tried to pair a second IKEA bulb of the same type, and that also connected fine.

The bulbs are on the latest firmware.

notenoughtech commented 2 years ago

If you open z2m log in the terminal after restarting the service you'll get the coordinator data including the firmware

On Fri, 7 Jan 2022, 12:40 flatsiedatsie, @.***> wrote:

I just did another test.

  • Current development version of ZIgbee2MQTT
  • Out-of-the-box Sonoff stick (unflashed). Don't know which version it's running (how can I tell?)

Before I disabled debugging I was able to pair an IKEA motion sensor.

The bulb connected normally and I can toggle it and change its brightness.

I then tried to connect a second IKEA bulb of the same type, and that also connected fine.

— Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee2mqtt/issues/9117#issuecomment-1007376954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKDRL225TRAOGYXE7Q4OUSDUU3NLVANCNFSM5FXV5MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

flatsiedatsie commented 2 years ago

@notenoughtech thanks, found it.

"coordinator":{"ieee_address":"0x00124b0024XXXXXX","meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20210708,"transportrev":2},"type":"zStack3x0"}

That seems to be the same revision as the one @TeslaCTL had issues with, version 20210708. Perhaps TeslaCTL could try testing the dev version of Z2M and see if that solves the pairing issues?

I also just tried to pair a Xiaomi temperature/humidity/pressure sensor, and it worked flawlessly again.

TeslaCLT commented 2 years ago

@flatsiedatsie I actually can only get things to pair if I put herdsman in debug mode (beyond that first bulb) -- but perhaps that is what you meant and misspoke? The command I provided (from another thread) turns on debug mode for herdsman.

Right now I have my 30 devices paired with the last 5 still on SmartThings. I'm going to wait a week or so to make sure the network is stable. I've noticed that my map currently has everything paired directly with the coordinator, though with sibling links to other routers. Of the 30 devices, only 1 isn't a router, but I'm still surprised to see none of them pairing with the other routers around them.

flatsiedatsie commented 2 years ago

I actually can only get things to pair if I put herdsman in debug mode

Yes I understood. That's why I did a test here without debug mode. The fact that is still worked for me without debug mode, though being on the dev version of Z2M, might indicate that the issue has already been solved in a newer version of Z2M?

Then again, it might be something else.

I'm still surprised to see none of them pairing with the other routers around them.

Interesting. My network graph usually shows a lot of routing over secondary devices (IKEA lightbulbs mostly).

TeslaCLT commented 2 years ago

Oh, you mean the dev branch of z2m. I misunderstood/misread and thought you meant the dev branch of the Sonoff firmware. I may indeed look into the dev branch of z2m then.

FYI, a day later, I still have the same pairing issues. I have to turn on debug mode to pair anything. So hopefully, as you said, the newer release will clear things up.

flatsiedatsie commented 2 years ago

Ah, sorry for the confusion.

oferwald commented 2 years ago

Same issue here, latest firmware (1217). Pairing is really hard, and it seems nearly impossible via a router. (I was able to pair a few devices directly to the coordinator, after some hard resets, but this is not the way). I could not pair devices from sonoff itself, such as a motion detector from the SNZB series. When activating debug mode and trying to pair, it seemed like there was a storm of messages, but the interview didn't even start. And I am with the latest-dev branch. @Koenkk I do have a CC2531 sniffer, what capture would you like, and what scenarios do you need me to sniff? When allow_join is true? when I enable a specific router to allow_join? I don't want to provide with useless data, and since the network here has 46 devices that were already paired. there is a lot of useless data in my captures. LMK.

u8915055 commented 2 years ago

Just to add to this thread.. and full disclaimer, i am using ZHA now but i wanted to share this because i am seeing a difference under 1217. I hope to go back to Z2M soon but was having too many issues. With ZHA I was able to pair about 25 sensors to it.. various ones from Sonoff, Aqara, Konke, etc. I have lots of sensors. I was able to get every single sensor to pair. some of them i had to pair a couple times to get the battery state to come up properly.. been stable ever since. However i will say the Sonoff ones are the hardest to pair. Took me several tries to get them to pair but eventually they do. I am finding aqara sensors especially pairing better with 1217. Overwhelmingly i find the tuya sensors pair the best.

Koenkk commented 2 years ago

@oferwald is your network fresh or did you migrate from a CC2531 to the sonoff stick without repairing devices?

oferwald commented 2 years ago

@Koenkk Migrated from the CC2531 without repairing devices, I did have to power off some devices that allowed them to connect with the new coordinator.

Koenkk commented 2 years ago

@oferwald this is probably the problem, see e.g. https://github.com/Koenkk/zigbee2mqtt/issues/8670. If you are experiencing pairing problems after migrating from a CC2530/CC2531 to a sonoff stick without repairing devices it is recommended to repair a few routers starting from those who are the closest to the coordinator.

oferwald commented 2 years ago

@Koenkk Ok, I will do that, any special re-pairing instructions? should I force them out of the network before re-pairing? Should I change any network params? change the keys or something?

This still does not explain why I am unable to pair the sonoff motion sensor when I am near the coordinator, or is it connected somehow? maybe because I originally tried to pair it near some routers?

Thanks for your patience

Koenkk commented 2 years ago

@oferwald you can repair them by just enabling joining in z2m and factory resetting the routers. Sonoff motion sensors are always hard to pair (even with a CC2531) so these are not the best reference.

oferwald commented 2 years ago

@Koenkk Hi, I tried factory reset for some tuya covers acting as routers, this is what I get: Zigbee2MQTT:warn 2022-01-09 12:52:09: Device 'Living room diagonal shutter' left the network dress":"0xbc33acfffe682925"},"type":"device_leave"}'

The device is not interviewing again, and when I take the power out and do a reset again, I get the same message again (although it has already left the network? how can it leave again?)

And now those few routers are out of my network, not joining again. :( something is really off here. any ideas?

Koenkk commented 2 years ago

@oferwald probably those routers try to join again via other routers where pairing is not working with.