Add migration path from existing zwave component

[cgarwood]

There's plenty more that needs done first, but I wanted to create an issue so it doesn't get lost...

We need to figure out some sort of migration path for users using the existing zwave component. Ideally we'd be able to migrate all of the entity_ids and device registry entries from the old zwave component to the new, so people don't have to rename/reconfigure them all.

---

• Migrate devices in device registry
• Migrate entities in entity registry
• Cleanup node entity

In case of normal Home Assistant including the supervisor:

• Offer user to migrate
• Auto initialise OZW Daemon add-on
• Auto initialise MQTT Broker add-on if needed
• Migrate Network key from config to OZW Daemon

In case of core-only Home Assistant:

• Offer user to migrate
• Show manual steps, like installing the OZW Daemon

[matejdro]

Is there a way to migrate manually?

I would like to help with the project, but I don't want to set up my whole z-wave network from scratch to do so.

[marcelveldt]

Well, you will not have to setup your Z-Wave network from scratch. The config is stored on your stick. What you do need to do is setup the devices/entities in HomeAssistant as there is no migration path yet.

[balloob]

So the trick for migration is going to be evolving around entity registry. It does require that given a unique ID of a Z-Wave entity, we can generate the unique ID that the entity would have with the new integration.

Migration will be destructive because an entity ID can only be registered to a single entry.

Migrate domain zwave -> zwave_mqtt:

• Go over all entity registry entries with domain = zwave
• If we can match it:
  • update entry with domain=zwave_mqtt, unique_id=<new_id>
  • Add identifier to device registry entry (if available).
• If we cannot match, remove it.

The Z-Wave integration has different unique ID implementations. For node entities they use node-<node_id>, otherwise they use <node_id>-<values.primary.id>

[marcelveldt]

@balloob what about keeping the same name ? So the new integration will be zwave instead of zwave_mqtt. That would solve part of the puzzle and the web UI will start working out of the box if we have all services in place.

[balloob]

We can't, as not everyone wants to upgrade and we can't migrate by just adding an mqtt broker and setting up the ozwdaemon.

[marcelveldt]

Valid point but do you plan to keep support of multiple integrations ? Or keep it side-by-side for a while and EOL the old implementation ? Good to know the plan upfront I guess.

But still back to the drawing board regarding migration. Somehow reinstating the previous ID's would be super user friendly.

[balloob]

The old one hasn't received upgrades in a while and I don't see anyone picking it up. We will promote this one and hide the other one.

[Fishwaldo]

It’s on my todo list to let you know what will be able to map directly between 1.4 and 1.6 and what may require some “magic” to map Into 1.6.

[marcelveldt]

@Fishwaldo will the ValueIDKey be the same when migrating from 1.4 to 1.6 ? Or is this something generated by OZW itself and not from the actual hardware/mesh ? If the ValueIDKeys stay the same, migration will be super easy.

[marcelveldt]

identifier for device in device-registry

current implementation:

first node_instance: identifier = (DOMAIN, node_id, node_name)

if node_instance > 1: identifier = (DOMAIN, node_id, instance, node_name)

node_name = node.name or f"{node.manufacturer_name} {node.product_name}"

Link to code

---

zwave_mqtt implementation:

device_id = "{ozw_instance}.{node.node_id}.{node_instance}"
identifier = (DOMAIN, device_id)

Link to code

---

First step of migration would be to migrate the device_id's in the current implementation to be similar with what we use now.

• ozw_instance fixed to 1.
• Strip device name.
• If instance_id is missing, set it to 1.

BTW: Very strange to use the device name as unique_id in the current implementation. Removing/replacing nodes is properly handled so safe to rely only on the node_id.

[marcelveldt]

unique id for entities

current implementation: f"{node.node_id}-{value.object_id}"

value.object_id == primary value valueid_key

Link to code

---

zwave_mqtt implementation

f"{self.primary.node.id}-{self.primary.value_id_key}"

Link to code

---

So assuming the ValueIDKey status/is the same between OZW 1.4 and 1.6 this unique id for the entity is the same. No migration needed, except for attaching it to the corrected device_id

[marcelveldt]

@balloob @MartinHjelmare I worked out some details for the migration path, see my comments above. You guys are more familiar with the device and entity registry but this sounds perfectly doable, right ? Just a matter of converting the device id's upfront.

The old implementation created a separate entity for the node itself with some attributes like battery_level. We do not create that entity anymore (values will be separate sensors now) so after migrating this will be a "dead" entity that we should maybe cleanup.

Source: https://github.com/home-assistant/core/blob/dev/homeassistant/components/zwave/node_entity.py

Updated the first post with some steps needed for the migration.

[cgarwood]

we should probably make the unique_id for this component include the openzwave instance in it as well, like the device registry. Otherwise, could we end up getting conflicts if there are 2 OZW instances in MQTT with similar devices that have overlapping node IDs?

[marcelveldt]

Yeah, that would be even more safe. I’m not sure how big the chance is to have the same ValueIDKey on multiple instances, let alone with matching node id

[Fishwaldo]

@Fishwaldo will the ValueIDKey be the same when migrating from 1.4 to 1.6 ? Or is this something generated by OZW itself and not from the actual hardware/mesh ?

Mostly.... with a few big corner cases.

1) Index Numbers went from byte (255 possible indexes) to Short (16364 possible indexes) and some Values were moved index numbers. 2) Alarm (Notification CC) - Totally Re-written. Not mapable. 3) UserCode CC - Totally re-written. Not Mapable. 4) Central Scene - Didn't really work in OZW 1.4. The "Hacks" that made it work were upto individuals so I don't know if there is a "standard" there.

There are probably a few others - Would there be a priority of values to migrate from certain CC's?

If you have the old ValueID Key - you can break it down to what it represents:

OZW 1.6 Mapping - https://github.com/OpenZWave/open-zwave/blob/9e288dfe02bbe6fa8f31ff26a067e2987c8c6367/cpp/src/value_classes/ValueID.h#L351

OZW 1.4 Mapping - https://github.com/OpenZWave/open-zwave/blob/449f89f063effb048f5dd6348d509a6c54fd942d/cpp/src/value_classes/ValueID.h#L288

[marcelveldt]

OK, we might be in luck in that case as the old implementation did not use (or generate entities for) 2, 3 and 4. As for item 1 we'll have to test if the valueId stays the same.

Thanks! BTW: Now we also have an answer to our previous questions about this ID.

[Fishwaldo]

Just a quick heads up - The Build Version is now reflected in the OZWDaemon_Version in the status topic - You can use this to check the version used.

I'm pointing this out - As a future version will have a change for some of the types of ValueID's exposed by certain CC - So you may want to think about how to do some conditional logic based upon the version of ozwdaemon/openzwave etc.

Also - Spelling Fix was just applied for the above

For your reference - Major.Minor bumps in versions on OpenZWave side reflect API and fundamental functionality changes. Minor Numbers are bug fixes/builds and should be API Stable. So you want to at least watch for OpenZWave 1.x (next release being 1.7) or ozwdaemon 0.x changes.