search

Koenkk / zigbee-herdsman

A Node.js Zigbee library
MIT License
456 stars 277 forks source link

ZNP Security Manager Table Occupancy Function Fix #393

Closed castorw closed 2 years ago

castorw commented 2 years ago

This PR fixes the security manager table occupancy function which caused APS keys not to be backed up properly. Some Z-Stack firmware versions use different empty states of the security manager entries, this caused the backup mechanism to identify an security manager entry for the first entry in address manager and not backup its key from TCLK table since there was an indication of arbitrary key being present in the APS link key table.

What to expect?

sjorge commented 2 years ago
  • If you have a network restored without APS keys where the routers expect encrypted APS communication, you will have to re-commission your network,

You can sort of get around it, by using join via device and re-pairing all the routers 1 by 1, it's slow AF but it's an option if you have stuff behind a wall for example. You can't use the generic permit join as then it's whack-a-mole via which router you end up going, which can fail if it's one of those without keys

castorw commented 2 years ago

@sjorge Yeah you are right. You can shutdown routers and re-pair them one by one. That should do the job and keep the rest of the network working.

sjorge commented 2 years ago

@sjorge Yeah you are right. You can shutdown routers and re-pair them one by one. That should do the job and keep the rest of the network working.

No need to power down, as you can spicy the router to use when enabling joining. It recently got fixed by Koen again.

castorw commented 2 years ago

@sjorge Okay then. @Koenkk we may provide information ok this in release notes.

sjorge commented 2 years ago

@sjorge Okay then. @Koenkk we may provide information ok this in release notes.

Yeah I ended up snaking out form the coordinator, did 2 bulbs nearby, then use those to get the in wall switch, then the hallways bulb, ... not fun but better than building from scratch as I got to keep all the pesky end devices online :)

Koenkk commented 2 years ago

I can confirm this indeed fixes the issue.

This is a sniff of pairing a device via a router after a reflash:

Screenshot 2021-07-23 at 14 37 12

The Update Device is now followed up by a Transport Key which allows to device to join, great work @castorw !

Koenkk commented 2 years ago

Added the following to the release notes:

Screenshot 2021-07-23 at 14 49 30
castorw commented 2 years ago

@Koenkk thanks!

Just to shed a bit more light on this. The primary reason this happens is the fact that having a R21+ spec coordinator and router causes the devices to negotiate an APS key and therefore both these devices may internally want to communicate in APS encrypted fashion. The routers from sniffs seen before indicated that the coordinator was sending only network-encrypted frames while the router joining the device kept pushing APS-encrypted payloads which weren't decipherable by the coordinator since it lost the APS link key after restore from backup - which was broken - because of the stuff fixed by this PR.

sjorge commented 2 years ago

@Koenkk you might want to make it clear in the notes they need to update z2m and at least have it started/stopped once before updating the coordinator with the new firmware. If they do it the other way around they lose the APS keys.

Koenkk commented 2 years ago

@sjorge good point, added this.

image

sjorge commented 2 years ago

Perhaps swap both bullet points too?

Koenkk commented 2 years ago

@sjorge done, thanks

copystring commented 2 years ago

My stick is already on 20210708. After updating to latest dev and restarting z2m do I have to re-flash the stick or is it ok like this?

sjorge commented 2 years ago

My stick is already on 20210708. After updating to latest dev and restarting z2m do I have to re-flash the stick or is it ok like this?

You probably have to remove and re-add all your routers, you should be able to do them one by one and use the join via device feature (you need to start at the coordinator obviously)

copystring commented 2 years ago

My stick is already on 20210708. After updating to latest dev and restarting z2m do I have to re-flash the stick or is it ok like this?

You probably have to remove and re-add all your routers, you should be able to do them one by one and use the join via device feature (you need to start at the coordinator obviously)

Hm. I think you misunderstood my question.

I understand the part about having to re-add/re-pair all my routers. In the notes from Koen it says I have to flash 20210708 after updating to the latest z2m and restarting it. My stick is already on 20210708. So do I have to flash 20210708 again after updating and restarting?

Let me know if I'm unclear about this...

sjorge commented 2 years ago

My stick is already on 20210708. After updating to latest dev and restarting z2m do I have to re-flash the stick or is it ok like this?

You probably have to remove and re-add all your routers, you should be able to do them one by one and use the join via device feature (you need to start at the coordinator obviously)

Hm. I think you misunderstood my question.

I understand the part about having to re-add/re-pair all my routers. In the notes from Koen it says I have to flash 20210708 after updating to the latest z2m and restarting it. My stick is already on 20210708. So do I have to flash 20210708 again after updating and restarting?

Let me know if I'm unclear about this...

No if you're already on 20210708 you should be good.

Koenkk commented 2 years ago

I noticed texas instruments released a new SDK, would be good to include this in the next fw update:

tested this firmware and looks good, @sjorge can you also give it a try?

EDIT: something seems wrong with this fw, got some MEM_ERORR, lets stick to 20210708 for the next release.

sjorge commented 2 years ago

Did you forget to push? It’s still the one from 2 weeks ago.

~ sjorge

On 24 Jul 2021, at 10:36, Koen Kanters @.***> wrote:

 I noticed texas instruments released a new SDK, would be good to include this in the next fw update:

tested this firmware and looks good, @sjorge can you also give it a try?

20210723: https://github.com/Koenkk/Z-Stack-firmware/tree/develop/coordinator/Z-Stack_3.x.0/bin — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Koenkk commented 2 years ago

@sjorge I've withdrawn this firmware since it caused memory errors (https://github.com/Koenkk/zigbee-herdsman/pull/393#issuecomment-886021164)

ahmaddxb commented 2 years ago

Perhaps swap both bullet points too?

where can i find this? could you paste the url, I would like to read the rest.

Koenkk commented 2 years ago

These are the release notes of the next release, not publicly available yet.