search

zigpy / bellows

A Python 3 project to implement EZSP for EmberZNet devices
GNU General Public License v3.0
182 stars 86 forks source link

[REQUEST] Support for Silicon Labs EFR32 (Mighty Gecko family) based Zigbee radio adapters using newer EZSP serial protocol? #243

Closed Hedda closed 3 years ago

Hedda commented 4 years ago

Can the bellows library be made to support newer EZSP serial protocol such as EZSP v6, v7, and v8 when used with ZNet 6.x on Silicon Labs EFR32 (Mighty Gecko family) based Zigbee radio adapters?

See ex. #242 as that hardware is based on this same family and series of chips from Silicon Labs.

Zigbee EmberZNet version 6.7.0.0 has breaking change (with EZSP Protocol Version 8 and Secure EZSP Protocol Version 2) as both EZSP and Secure EZSP have adopted a new frame format with the following changes: (1) the fields of "Frame Control" and "Frame ID" are now two bytes; (2) no longer use "Legacy Frame ID"; (3) consume two bits of "Frame Control" to indicate the frame format version which is version 1 now.

Perhaps newer modules will dethrone Nortek HUSBZB-1 for ZHA as the must-have Zigbee adapter?

Adminiuga commented 4 years ago

Yeah, without a matching TC Link key, devices won't be able to get the network key.

s-hadinger commented 4 years ago

I also realized that I set rxRadioPower to 1 because I had no clue about the value. I found in the doc that for EFR32 it should be 20, this also explains the erratic behavior.

Now if I can benefit from you experience:

I paired an OSRAM switch mini, but didn't receive any childJoinHandler message. I recevied the following and pairing seemed to work:

But, with a Xiaomi device, I get the following:

Do you have any idea why I don't receive childJoinHandler with OSRAM, and why Xiaomi device leaves? In parallel I will try to set up a Zigbee sniffer.

Edit: I realize that the OSRAM didn't pair, but registered itself as a neighbour. I confirmed it with getNeighbour and it does have an entry. I'm a bit lost here...

Adminiuga commented 4 years ago

childJoinHandler is when an EndDevice joins directly to NCP. trustCenterJoinHandler is the handler notifying about a device being joined. If OSRAM is not an EndDevice, you won't get the childJoinedHandler

s-hadinger commented 4 years ago

Ah got it. The CC2530 Z-Stack I'm used to is of higher level interface. Since the OSRAM wall switch is also a router, it is considered as a neighbour, not a child. This does not explain why the Xiaomi leaves after 2 minutes, but thanks. I'm progressing.

Hedda commented 4 years ago

@NilsBohr you think Elelabs might be willing to donate a few EFR32MG13P based adapters to bellows and zigbee-herdsman devs?

Thinking that such sponsoring will benefit Elelabs as a company since more developers working on this will probably lead to more users buying your Silicon Labs based adapters as long as you also provide firmware compatible with those open source projects.

bellows is primarily used by Home Assistant's ZHA integration, and zigbee-herdsman is used by Zigbee2mqtt and IoBroker.

Hello guys,

That's Kit from Elelabs.

I thought that Elelabs had not published exactly which chip type they were using other than they were from Silicon Labs, so I had no idea that they are based on EFR32 MG1 (EFR32 Mighty Gecko Series 1).

Jumping here in the discussion, if you don't mind.

We had sold:

  • EZBUSBA model using version EM357
  • ELU012 model using EFR32MG1 Now we are selling ELU013, which is using EFR32MG13P.

The Raspberry shield also had different models:

  • EZBPIS based on EM357
  • ELR022 based on EFR32MG1B232 (32Kb RAM) Now we are selling ELR023, which is also based on EFR32MG13P732 (64Kb RAM)
NilsBohr commented 4 years ago

@Hedda

Yes, we would be glad to. Please reach out to me at sne@elelabs.com

Hedda commented 4 years ago

@NilsBohr you think Elelabs might be willing to donate a few EFR32MG13P based adapters to bellows and zigbee-herdsman devs?

Thinking that such sponsoring will benefit Elelabs as a company since more developers working on this will probably lead to more users buying your Silicon Labs based adapters as long as you also provide firmware compatible with those open source projects.

bellows is primarily used by Home Assistant's ZHA integration, and zigbee-herdsman is used by Zigbee2mqtt and IoBroker.

Yes, we would be glad to. Please reach out to me at sne@elelabs.com

I am not a skilled enough programmer myself, but maybe @Adminiuga @puddly @dmulcahey & @s-hadinger are interested(?) 👍

Perhaps @rcloran @AndreasBomholtz & @damarco would also be interested in getting back to developing bellows and zigpy?

PS: @NilsBohr Suggest that you also post the same offer in the zigbee-herdsman discussion here -> https://github.com/Koenkk/zigbee-herdsman/issues/168

Hedda commented 4 years ago

Silabs just released a new $99 "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) which only contain one mainboard and two low-power radios (Zigbee Green Power compatible):

(Google search shows that relatively inexpensive kit can already be bought from resellers globally).

While such low-power radios are not really usable as Zigbee coordinators, for $99 that kit can be useful to developers if buying such an official kit also gives full access to the Zigbee Stack/SDK and libraries.

FYI, those who are interested in the official $99 SLWSTK6021A started kit might also be interested in buying article SLWRB4180A for around $49 at the same time as that is the matching EFR32MG21 based high-power (+20 dBm) radio board for that dev board kit.

https://www.silabs.com/products/development-tools/wireless/slwrb4180a-efr32-wireless-gecko-radio-board

(Google search shows that SLWRB4180A can be bought from many of resellers globally who also stock the SLWSTK6021A kit).

Adminiuga commented 4 years ago

This does not explain why the Xiaomi leaves after 2 minutes, but thanks. I'm progressing

👍 if you figure this one out, let me know, as i'm troubleshooting the same issue. Have two EZSP running 6.3.3 version, the dev works fine, but it is only a few devices, but in prod, ZB3 joins, stays a bit (works fine during that period) and then leaves. Both prod and dev running the same version, just number of devices is different. I have strong suspicion it might be related to some ZB3 settings/policy settings, as downgrading to protocol version 6 or 5 fixes this issue.

Is the Xiaomi a ZB3 device?

s-hadinger commented 4 years ago

@Adminiuga I doubt the Xiaomi runs ZB3. When I was using CC2530, it was a ZB1.2 network. I need to re-read the doc carefully to make it ZB1.2 devices compatible. This means sending the network key in clear, and pre-registering the "well-known" Link Key ZigBeeAlliance09. It looks from the doc that you need to add it explicitly.

I will also try a configuration without Trust Center. It's not ideal but for domestic Zigbee networks as we use in Tasmota, that's enough to start with.

Edit: can you please share all the security settings you are using to configure EZSP?

Adminiuga commented 4 years ago

This means sending the network key in clear

Well, technically it is encrypted by the tc link key, but since the link key is well known you might as well send it in the clear.

I can show EZSP settings configured for bellows

020-06-24 10:53:42 DEBUG (MainThread) [bellows.ezsp] Resetting EZSP
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command version: (4,)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 0 (version) received: b'07023066'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command version: (7,)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 0 (version) received: b'07023066'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Switched to EZSP protocol version 7
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] EZSP Stack Type: 2, Stack Version: 26160
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_INDIRECT_TRANSMISSION_TIMEOUT: 18>, 7680)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_TRUST_CENTER_ADDRESS_CACHE_SIZE: 25>, 2)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_PAN_ID_CONFLICT_REPORT_THRESHOLD: 34>, 2)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_MAX_END_DEVICE_CHILDREN: 17>, 24)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_KEY_TABLE_SIZE: 30>, 4)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_SECURITY_LEVEL: 13>, 5)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_SUPPORTED_NETWORKS: 45>, 1)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_SOURCE_ROUTE_TABLE_SIZE: 26>, 16)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_STACK_PROFILE: 12>, 2)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_APPLICATION_ZDO_FLAGS: 42>, 3)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_MULTICAST_TABLE_SIZE: 6>, 16)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_ADDRESS_TABLE_SIZE: 5>, 16)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_END_DEVICE_POLL_TIMEOUT: 19>, 8)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConfigurationValue: (<EzspConfigId.CONFIG_PACKET_BUFFER_COUNT: 1>, 255)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 83 (setConfigurationValue) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command getConfigurationValue: (<EzspConfigId.CONFIG_APS_UNICAST_MESSAGE_COUNT: 3>,)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 82 (getConfigurationValue) received: b'000a00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.zigbee.application] APS_UNICAST_MESSAGE_COUNT is set to 10
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command addEndpoint: (1, 260, 48879, 0, 0, 1, [], [1280])
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 2 (addEndpoint) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.zigbee.application] Ezsp adding endpoint: [<EzspStatus.SUCCESS: 0>]
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setConcentrator: (False, <EmberConcentratorType.HIGH_RAM_CONCENTRATOR: 65529>, 600, 1800, 2, 5, 0)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 16 (setConcentrator) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.zigbee.application] Set concentrator type: [<EmberStatus.SUCCESS: 0>]
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command networkInit: ()
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 23 (networkInit) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 25 (stackStatusHandler) received: b'90'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command getNetworkParameters: ()
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 40 (getNetworkParameters) received: b'000105fc01bf820a160685ee080f0000000000f8ff07'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setPolicy: (<EzspPolicyId.TC_KEY_REQUEST_POLICY: 5>, <EzspDecisionId.GENERATE_NEW_TC_LINK_KEY: 82>)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 85 (setPolicy) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setPolicy: (<EzspPolicyId.APP_KEY_REQUEST_POLICY: 6>, <EzspDecisionId.ALLOW_APP_KEY_REQUESTS: 97>)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 85 (setPolicy) received: b'00'
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Send command setPolicy: (<EzspPolicyId.TRUST_CENTER_POLICY: 0>, <EzspDecisionId.ALLOW_PRECONFIGURED_KEY_JOINS: 1>)
2020-06-24 10:53:43 DEBUG (MainThread) [bellows.ezsp] Application frame 85 (setPolicy) received: b'00'

you can ignore the endpoint addition and setting the concentrator type. Most of these settings were carried over from the original versions of bellows, which I think were established during protocol version 4 or V5

s-hadinger commented 4 years ago

Thanks this is helpful. I'm not sure why we need the setConcentrator(False,...) call.

The log you provided was from an already existing network networkInit. I'm more interested in the security settings you needed to setup before calling 'formNetwork` in the first place.

Adminiuga commented 4 years ago

setConcentratorType is for sending out MTORR messages, at least this is what I think it does.

here's forming the network

2020-06-24 11:34:52 DEBUG (MainThread) [bellows.ezsp] Send command networkInit: ()
2020-06-24 11:34:52 DEBUG (MainThread) [bellows.ezsp] Application frame 23 (networkInit) received: b'93'
2020-06-24 11:34:52 DEBUG (MainThread) [bellows.ezsp] Send command setInitialSecurityState: (<EmberInitialSecurityState bitmask=2820 preconfiguredKey=[90, 105, 103, 66, 101, 101, 65, 108, 108, 105, 97, 110, 99, 101, 48, 57] networkKey=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2] networkKeySequenceNumber=0 preconfiguredTrustCenterEui64=00:00:00:00:00:00:00:00>,)
2020-06-24 11:34:52 DEBUG (MainThread) [bellows.ezsp] Application frame 104 (setInitialSecurityState) received: b'00'
2020-06-24 11:34:52 DEBUG (MainThread) [bellows.ezsp] Send command formNetwork: (<EmberNetworkParameters extendedPanId=00:00:00:00:00:00:00:00 panId=0x5ae4 radioTxPower=8 radioChannel=15 joinMethod=EmberJoinMethod.USE_MAC_ASSOCIATION nwkManagerId=0x0000 nwkUpdateId=0 channels=Channels.CHANNEL_25|CHANNEL_20|CHANNEL_15>,)
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Application frame 30 (formNetwork) received: b'00'
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Application frame 25 (stackStatusHandler) received: b'90'
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Send command setValue: (<EzspValueId.VALUE_STACK_TOKEN_WRITING: 7>, 1)
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Application frame 171 (setValue) received: b'00'
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Send command getNetworkParameters: ()
2020-06-24 11:34:53 DEBUG (MainThread) [bellows.ezsp] Application frame 40 (getNetworkParameters) received: b'0001bc1126bc0c962cd2e45a080f0000000000f8ff07'

just replace the network key. The source is here https://github.com/zigpy/bellows/blob/6e66bffe8d49a6a36f41a12b2aebdf1ed7ee384e/bellows/zigbee/application.py#L201-L226 make a call to https://github.com/zigpy/bellows/blob/6e66bffe8d49a6a36f41a12b2aebdf1ed7ee384e/bellows/zigbee/util.py#L8-L32

Keep in mind this is all for the older version and might need adjustments for V8

mtx512 commented 4 years ago

@s-hadinger I looked at some code there are options on the policy set allow re-joins with a well known key

EZSP_TC_REJOINS_USING_WELL_KNOWN_KEY_POLICY set it to 1

there also then a rejoin timeout value EZSP_CONFIG_TC_REJOINS_USING_WELL_KNOWN_KEY_TIMEOUT_S . However may be this can set to 0.

You can set the well known key (0x5A, 0x69, 0x67, 0x42, 0x65, 0x65, 0x41, 0x6C 0x6c 0x69, 0x61, 0x6E, 0x63, 0x65, 0x30, 0x39 } using addTransientLinkKey.

s-hadinger commented 4 years ago

Thanks, but it's still not fully functional.

Here is what I set up, trying to mimck the relaxed security of ZB1.2:

Using EZSP_TC_REJOINS_USING_WELL_KNOWN_KEY_POLICY does not seem necessary.

The Xiaomi joins and the TC sends the key in clear.

During these 2 minutes, I don't receive any readings from the device, so I'm not sure it actually paired. I will try to send ZDO requests, but I've just discovered that I need to manually build the frames (ZStack had an API for that).

@Adminiuga Do you know why there is setValue: (<EzspValueId.VALUE_STACK_TOKEN_WRITING: 7>, 1)? I will try tomorrow with Security Policy set to 0x0B04 like in your example.

Adminiuga commented 4 years ago

Do you know why there is setValue: (<EzspValueId.VALUE_STACK_TOKEN_WRITING: 7>, 1)

Hrm, I thought that was a leftover from one of my tests, but it is not. I'm not positive on the exact purpose, but per image I'm thinking it is supposed to retain settings like pan-id, extended pan id etc related to stack 🤷

Adminiuga commented 4 years ago

Any chance you could try with some me other than xiaomi brand? With xiaomi you never know if something is wrong on your side or is it xiaomi interpretation of the standard. I'd recommend any device from centralite

Hedda commented 4 years ago

I doubt the Xiaomi runs ZB3. When I was using CC2530, it was a ZB1.2 network.

FYI; I believe that only a few newer Xiaomi / Aqara products released after 8th July 2019 support the standard Zigbee 3.0 specification. Not sure if Xiaomi / Aqara release Zigbee 3.0 compatible firmware updates to any older devices released before then. Sources are Xiaomi / Aqara announcement: https://news.mydrivers.com/1/634/634160.htm and https://news.mydrivers.com/1/635/635172.htm plus the fact that they also released their new "Xiaomi Multimode Gateway" (a.k.a. "Xiaomi Smart Gateway V3") in December of 2019 which is marketed as a Zigbee 3.0 gateway.

I'm sure you also know this, but for reference, it should be noted that almost all Zigbee 3.0 devices are backwards compatible with ZHA 1.2 (Zigbee Home Automation 1.2 profile) and/or are backwards compatible with ZLL 1.0 (Zigbee Light Link 1.x profile) and with connect using that profile if your Zigbee network established by your Zigbee coordinator is ZHA 1.2

I'm sure you also know this too, but again for reference, if you run Texas Instruments Z-Stack Home 1.2 (and not Z-Stack 3.0.x) on your CC2530 then can only support ZHA 1.2 and thus only be compatible with Zigbee devices in ZHA 1.2 mode (therefor forcing ZHA 1.2 backwards compatible if they support it). So you need to run a Z-Stack 3.0.x on your CC2530 if you want it to be compatible with Zigbee 3.0

MattWestb commented 4 years ago

Very easy Xiaomi have only 2 certified products !!!! https://zigbeealliance.org/zigbee_products/?product_type=certified_product&se=Xiaomi+

The big things its compatibility for LL and HA and the large are Philips hue and Ikea Tradfri for the most users. Both have good and bad things but wildly used. Xiaomi making great sensors but all (-1) its home cocked and not following any standard only the Xiaomi one.

Hedda commented 4 years ago

Very easy Xiaomi have only 2 certified products !!!! https://zigbeealliance.org/zigbee_products/?product_type=certified_product&se=Xiaomi+

No they have more but they are both soley marketed and certified under the "Aqara" brand (and not under "Xiaomi" brand), see:

https://zigbeealliance.org/zigbee_products/?product_type=certified_product&se=Aqara

MattWestb commented 4 years ago

You are right "Lumi United Technology" have 8 and Xiaomi have 2 so all together 10 (of ??).

Look for Philips, Osram and Ikea you get little more "certified devices"

Adminiuga commented 4 years ago

Interesting, those aqara look exactly like the old models and I thought they never planned to update those. Did anyone ordered those recently and confirmed they are ZB3?

s-hadinger commented 4 years ago

Thanks for you help. I tried with 2x Xiaomi sensors (non ZB3), an OSRAM mini-switch, and a BlitzWolf TS0201. They all behave the same: they start the pairing process, are accepted by the TC, but then nothing happens and they are rejected after 2 minutes. Non of them are ZB3.

I feel I'm missing a tiny piece here. This is due to my ignorance of Zigbee protocol. Do I need to send any ZDO message to finalize the pairing of an end-device?

Note: I also tried to disable security, it's still the same behavior: pairing then leaving after 2 minutes (except I don't get the TC approval message since there is no TC in this variant).

My next step will be to get a Zigbee sniffer after all.

Adminiuga commented 4 years ago

To cover all bases, order a zigbee 3.0 device, maybe some Konke sensor or new xiaomi zb3 illuminance sensor.

I don't think protocol mandates any ZDO messages to be sent, but with that said I guess it is up to the vendor to decide whether to stay on the network or not, if nothing bounds any clusters. My expectation this shouldn't be required. And HA1.2 xiaomi/aqara sensor really don't care, as they are hardcoded to send attribute updates to 0x0000 NWK address, without any configuration.

Hedda commented 4 years ago

Interesting, those aqara look exactly like the old models and I thought they never planned to update those. Did anyone ordered those recently and confirmed they are ZB3?

We can probably assume that those devices are based on the same hardware but now have new firmware with Zigbee 3.0 stack, or?

I have not bought any ew Xiaomi/Aqara device myself, however, to get on that "CERTIFIED BY THE ALLIANCE" list on zigbeealliance.org means that they been sent in and independently certified as Zigbee 3.0 compatible by the Zigbee Alliance, so consider just being on that list that as confirmed to follow the Zigbee 3.0 specification.

Hedda commented 4 years ago

To cover all bases, order a zigbee 3.0 device, maybe some Konke sensor or new xiaomi zb3 illuminance sensor.

If you have CC2530 module or device like Sonoff BASIC ZBR3 then could try DIYRuZ/DIYRuZRT Zigbee 3.0 firmware (by @kirovilya ):

My next step will be to get a Zigbee sniffer after all.

Adminiuga might recommend a better solution for making your own Zigbee sniffers, however, if you will be using Wireshark for Zigbee sniffing then check out https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html for Texas Instruments CC2531 based Zigbee sniffer and https://github.com/zsmartsystems/com.zsmartsystems.zigbee.sniffer for Silicon Labs Ember based Zigbee sniffer.

s-hadinger commented 4 years ago

Yes. I have a spare cc2531 that I will flash with a Zigbee sniffer stack

Adminiuga commented 4 years ago

You can also look at alternative implementation from ZSmartSystems, here's init https://github.com/zsmartsystems/com.zsmartsystems.zigbee/blob/23ec347971d0be10e9699859832d2160364b3c05/com.zsmartsystems.zigbee.dongle.ember/src/main/java/com/zsmartsystems/zigbee/dongle/ember/ZigBeeDongleEzsp.java#L282

And here's I think is the network formation: https://github.com/zsmartsystems/com.zsmartsystems.zigbee/blob/23ec347971d0be10e9699859832d2160364b3c05/com.zsmartsystems.zigbee.dongle.ember/src/main/java/com/zsmartsystems/zigbee/dongle/ember/internal/EmberNetworkInitialisation.java#L103

s-hadinger commented 4 years ago

Thanks, that's useful. I reviewed the code and didn't spot anything different from what I'm currently doing. I will next try it with my EFR32.

s-hadinger commented 4 years ago

Thanks to all, I have now reached an important milestone. All basic functions are now implemented (except bindings) and the Tasmota version for EZSP v8 is now open for testing. I'm happy to share any learnings.

https://github.com/arendst/Tasmota/pull/8839

s-hadinger commented 4 years ago

I'm requesting the hive's knowledge.

I configured an OSRAM switch binding to group (multicast) addresses. Everything works well but I don't receive the content of the multicast message at coordinator level. I only receive a messageSentHandler (0x3F) callback for the multicast message with correct group address and endpoint.

Does anyone know how I can ask EmberZNet to send an incomingMessageHandler for a multicast message?

Adminiuga commented 4 years ago

you could try subscribing NCP to the multicast address, although I don't remember if it going to loopback it. check setMulticastTableEntry EZSP command

sprut666666 commented 4 years ago

@sprut666666 as mentioned in #242 is also working on similar EFR32MG1 based adapter for @sprut

The next batch will be based on the https://www.silabs.com/wireless/zigbee/efr32mg21-series-2-socs/device.efr32mg21a010f1024im32 and the latest SDK.

s-hadinger commented 4 years ago

@Adminiuga indeed but I would like to avoid it. The NCP does receive the message anyways. I'm looking for a way to surface it to the EZSP layer.

Adminiuga commented 4 years ago

Mind if I ask what are you trying to achieve? You have the message you are sending out, why do you need to hear it back ?

s-hadinger commented 4 years ago

Sure, I configure a remote OSRAM switch (On/Off) to control a bunch of zigbee devices via a group address. I'm running the EFR32 as a coordinator and would like to report all the group messages to a central MQTT broker (already works like this with CC2530).

The multicast message originates from an OSRAM switch. Looking with a Zigbee sniffer:

If I configure setMulticastTableEntry I do get the incoming multicast message from the OSRAM device, but reported as EMBER_INCOMING_UNICAST as well as a second synthetic message from the coordinator to itself.

Weird...

Adminiuga commented 4 years ago

Hrm, this is weird. with setMulticastTableEntry you are supposed to receive all those multicast messages and the type should be of Multicast one.

Adminiuga commented 4 years ago

here's a multicast from Ikea 5 button remote

2020-07-08 15:32:20 DEBUG (MainThread) [bellows.ezsp] Application frame 69 (incomingMessageHandler) received: b'020401060001ff000100000effba6075ffff0301040202'
2020-07-08 15:32:20 DEBUG (MainThread) [bellows.zigbee.application] Received incomingMessageHandler frame with [<EmberIncomingMessageType.INCOMING_MULTICAST: 2>, <EmberApsFrame profileId=260 clusterId=6 sourceEndpoint=1 destinationEndpoint=255 options=256 groupId=0 sequence=14>, 255, -70, 0x7560, 255, 255, b'\x01\x04\x02']
2020-07-08 15:32:20 DEBUG (MainThread) [zigpy.zcl] [0x7560:1:0x0006] ZCL deserialize: <ZCLHeader frame_control=<FrameControl frame_type=CLUSTER_COMMAND manufacturer_specific=False is_reply=False disable_default_response=False> manufacturer=None tsn=4 command_id=2>
dmulcahey commented 4 years ago

Thanks to all, I have now reached an important milestone. All basic functions are now implemented (except bindings) and the Tasmota version for EZSP v8 is now open for testing. I'm happy to share any learnings.

arendst/Tasmota#8839

How did you get past the key issue with devices joining?

s-hadinger commented 4 years ago

@Adminiuga I double checked, and I do receive a correct multicast message. The unicast was a side effect from my code. So all good. I'm just disappointed that you can't listen to all multicast addresses.

@dmulcahey What's working for me:

MattWestb commented 4 years ago

A very happy BillyGecko gratings !!

Welcome to minicom 2.7.1

OPTIONS: I18n                                                                
Compiled on May  3 2018, 15:20:11.                                           
Port /dev/ttyUSB1, 14:13:41                                                  

Press CTRL-A Z for help on special keys                                              

11 1a c2 02 8b c2 8a 7e
MattWestb commented 4 years ago

The Ikea module ICC-A-1 with the EFR32MG1P132F256GM32 its working in HA then loaded with EZSP 6.6.4.0 ICC-A-1 running EZSP in HA

I see that the 6.7.0.0 support its committed (#272). How its the status ? I was trying it and HA have the comport initiated but its looks like the NCP its not initiated. I think HA has not committed the last zigpy commitments in there docker image.

Adminiuga commented 4 years ago

in #272 it just EZSP version 8 framing supported. I have not tried it yet personally. I was trying to get the EBYTE module working, but no luck with it yet. Can flash it fine, but not getting any response in the terminal.

And I'm still in the process of refactoring bellows to offer more flexibility in supporting different protocol versions, which is a bit of PITA

MattWestb commented 4 years ago

Thanks for replay. I was not doing andy deep digging then the PR was looking like you is saying only frame adjustment. If you don't getting any reply for the NCP it's probably crashed. My experience its that then the bad NCP is crashing its very hard and not possible connecting with SWD and must boot in the bootloader for SWD. I have trying 3 different and the best maker of new NCP is grobasoz. Ask him if hi can do one for your module and you don't wasting your time (that it's better spend on bellow). Hi have made the working one for Sonoff and for the Ikea modul for my (both the 6.6.4.0 and 6.7.4.0). In meantime i trying little more serious testing with HA with the 6.6.4.0 EZSP.

PS: The EZSP its very very fast comparing with TI-CC253X and I think it's possible better then my RaspBee I

MattWestb commented 4 years ago

If you have one Ikea in the near and can soldering you can do a "billy" EZSP in some minutes and i have the working bootloader and NCPs. The On/Off dimmer E1743 it's only costing 6€ and nearly beginner skills it's needed. It being around the same performance as the EBYTE module. (You need one SWD / J-tag debugger for flashing on new bootloader)

tube0013 commented 4 years ago

I've had no luck with the EBYTE module. as far as the v8 support here are the logs with the reframing patch - creating a network with the sonoff zigbee bridge over the socket:// connection and joining the first device:

https://paste.ubuntu.com/p/zDBWTnJHJQ/

Here is what happens when the bridge is disconnected/rebooting:

https://paste.ubuntu.com/p/fy3d97kvrz/

Adminiuga commented 4 years ago

Logs look good.

tube0013 commented 4 years ago

I redid the network this am to get clean logs. but over night through the morning it was stable with the 4 devices on it.

MattWestb commented 4 years ago

Perhaps its the problem with the EFR32 gen 1 NCP firmware that not working well as I was having problem (was unpatched dev) but for @tube0013 it was working on the sonoff that is one EFR32 gen 2. I'm running my ZHA in docker and is for the moment to lazy for patching and compiling so cant helping debugging with my EFR32 gen 1. My firmware coming from the same maker of the working one for sonoff and have the same internal settings. Then i getting my hands on one docker images with the v8 patch i can also trying helping Adminiuga with testing. Thanks you both for working to getting it working.

tube0013 commented 4 years ago

I ran into issues with the TrustCenter joining devices after removing everything and trying to add again for the 4 or 5th time. Reflashed the EFR32 and all working again.