Supporting more network and node information in v2 spec

puddly commented 2 years ago

The primary motivation behind this change is to update the specification to support all of the extra data encoded by zigpy.state, utilized by EZSP and deCONZ coordinators:

device logical types, including the current node
current node's NWK address
NWK manager address
trust center link key

Other changes:

All encoded byte sequences are lowercase.
Since more information about the current node is stored, coordinator-centric keys have been renamed and grouped into a top-level node key.
null NWK addresses are disallowed. Instead, IEEE addresses are mapped to NWK addresses in a separate, top-level addresses key. If a node's NWK address isn't known, it's not stored in the cache.

Current example adapted to v2 spec:

{
  "metadata": {
    "format": "zigpy/open-coordinator-backup",
    "version": 2,
    "source": "zigbee-herdsman@0.13.65",
    "internal": {
      "date": "2021-02-08T19:35:24.761Z",
      "znpVersion": 2
    }
  },
  "stack_specific": {
    "zstack": {
      "tclk_seed": "4973f5450493c3cd90552c5f1d388a12"
    }
  },
  // Node info
  "node": {
    "ieee": "00124b0009d80ba7",
    "nwk": "0000",
    "type": "coordinator"
  },
  // Network info
  "network": {
    "pan_id": "cd0a",
    "extended_pan_id": "00124b0009d69f77",
    "update_id": 0,
    "security_level": 5,
    "channel": 21,
    "channel_mask": [
      21
    ],
    "network_key": {
      "key": "1a0355fd7abb5dc588a34998f8ee1233",
      "sequence_number": 0,
      "frame_counter": 108522
    },
    "aps_link_key": "5a6967426565416c6c69616e63653039",
    "nwk_manager": "0000"
  },
  // Device info
  "devices": [
    {
      "ieee": "04cf8cdf3c79455f",
      "type": "end-device",
      "is_child": false,
      "link_key": {
        "key": "9c88e969f3d3d23cd557304f2ac9cb97",
        "rx_counter": 6,
        "tx_counter": 10098
      }
    },
    {
      "ieee": "00124b002226ef87",
      "type": "end-device",
      "is_child": true,
      "link_key": {
        "key": "4422b6772c140f380dfd6f51f50e1693",
        "rx_counter": 842,
        "tx_counter": 35830
      }
    },
    {
      "ieee": "00158d0002ec571d",
      "type": "end-device",
      "is_child": true
    },
    {
      "ieee": "00158d0004506f9a",
      "type": "end-device",
      "is_child": true
    },
    {
      "ieee": "680ae2fffeae5647",
      "type": "router",
      "is_child": false,
      "link_key": {
        "key": "184b9674edab799d02523d3d32725f44",
        "rx_counter": 4170,
        "tx_counter": 370
      }
    },
    {
      "ieee": "0f01020304050607",
      "type": "router",
      "is_child": false,
      "link_key": {
        "key": "f1f2f3f4f5f6f7f8a1a2a3a4a5a6a7a8",
        "rx_counter": 20000,
        "tx_counter": 60010
      }
    }
  ],
  // Cached NWK addresses
  "addresses": {
    "04cf8cdf3c79455f": "1ad9",
    "00124b002226ef87": "6b5d",
    "00158d0002ec571d": "1ea2",
    "00158d0004506f9a": "3283",
    "680ae2fffeae5647": "06cb",
    "0f01020304050607": "270f"
  }
}

Thoughts? Better names for keys?

Hedda commented 1 year ago

@puddly @castorw Any updates on this? This may also be relevant in zigbee-herdsman as well as for deCONZ/Phoscon now too?

https://github.com/dresden-elektronik/deconz-rest-plugin-v2/issues/12

https://github.com/Koenkk/zigbee-herdsman/issues/665

https://github.com/Koenkk/zigbee-herdsman/issues/319

https://github.com/Koenkk/zigbee-herdsman/issues/242

Hedda commented 1 year ago

Any updates on this? This may also be relevant in zigbee-herdsman as well as for deCONZ/Phoscon now too?

dresden-elektronik/deconz-rest-plugin-v2#12

Koenkk/zigbee-herdsman#665

FYI, ChrisHae from Dresden Elektronik said he plan to add support to Dresden Elektronik's deCONZ/Phoscon's applications/APIs:

https://github.com/dresden-elektronik/deconz-rest-plugin-v2/issues/12

That is, he plans on adding export and import compatibility for "Open ZigBee Coordinator Backup Format" to deCONZ/Phoscon:

https://phoscon.de/en/conbee2/software#deconz

https://phoscon.de/en/conbee2/software

https://github.com/dresden-elektronik/deconz-rest-plugin

https://github.com/dresden-elektronik/deconz-rest-plugin-v2

puddly commented 1 year ago

@Hedda The file format does not store information about devices, it's just a coordinator state snapshot. Thus, when you write:

This would allow users to easily migrate to and from different Zigbee gateway applications without having to re-pair any of their devices.

This is not true and pretty misleading.

There is no way to migrate between any integrations, including ZHA and Z2M, because their on-disk device databases store completely different information and are incompatible. The best that can be done is for network settings to be re-used, which effectively treats every unknown devices a as new join. This won't pick up most battery-powered devices, only routers, so it is not a migration.

Thus, I don't believe there is any benefit from other projects utilizing this specification, as zigpy internally uses this v2 format.

zigpy / open-coordinator-backup

Supporting more network and node information in v2 spec #9