[Bug]: homekit devices with `wait for setup` not persistent across node-red restart/redeploy

sjorge commented 2 years ago

NRCHKB Plugin Version

1.4.3

Node JS Version

16.13.0

NPM Version

8.1.0

Node-RED Version

2.1.3

Operating System

OmniOS Bloody (updated 2 days ago)

What happened?

I saw that #393 landed a while ago so I wanted to simplify my setup a lot by embedding the homekit node inside a subflow.

This seemed to work at first glance but on redeploy all the devices in homekit end up in the default room again and dissapear from all automation and scenes that had included then.

Because the service node is inside the subflow, I am sending it a setup payload to configure it. My initial assumption was I was sending different setup payloads but on closer inspection this was not the case:

deploy 1

{"payload":{"nrchkb":{"setup":{"name":"Office Mood Light","manufacturer":"Philips","model":"Hue white and color ambiance E26/E27 1600lm","serialNo":"1x0000000000010114","firmwareRev":"1.88.2","hardwareRev":"","softwareRev":"20210422","characteristicProperties":"{\"On\":true,\"Brightness\":true,\"Hue\":true,\"Saturation\":true}"}}},"_msgid":"114a8797096f03d7"}

deploy 2

{"payload":{"nrchkb":{"setup":{"name":"Office Mood Light","manufacturer":"Philips","model":"Hue white and color ambiance E26/E27 1600lm","serialNo":"1x0000000000010114","firmwareRev":"1.88.2","hardwareRev":"","softwareRev":"20210422","characteristicProperties":"{\"On\":true,\"Brightness\":true,\"Hue\":true,\"Saturation\":true}"}}},"_msgid":"a787f3229af43d12"}

deploy 3

{"payload":{"nrchkb":{"setup":{"name":"Office Mood Light","manufacturer":"Philips","model":"Hue white and color ambiance E26/E27 1600lm","serialNo":"1x0000000000010114","firmwareRev":"1.88.2","hardwareRev":"","softwareRev":"20210422","characteristicProperties":"{\"On\":true,\"Brightness\":true,\"Hue\":true,\"Saturation\":true}"}}},"_msgid":"dabaee1eb90ea64f"}

So I'm not really sure if I am doing something wrong or if there is a bug somewhere.

How to reproduce?

Attached is a flow.json, this includes my subflow, and one instance of it. You might need to edit the IIEEE/groupid of the instance and hook it up to mqtt, in my case I am talking to zigbee2mqtt and also fetch generate the setup payload from the data.

flows.json.txt

Expected behavior:

The node to be persistent across deploys/node-red restarts if I feed it the same setup data.

Additional comments?

Additional comments here, if any.

Relevant log output

No response

sjorge commented 2 years ago

I was able to get the same behavior without a subflow involved or the need for mqtt.

This adds 2 service nodes + 4 inject nodes with a startup set to 5 and 6 to setup 2 bulbs.

If you deploy this flow they show up under default room, now move them to a different room and and restart node-red or redeploy the flow. The nodes disappear and reappear a few seconds later in the Default Room.

[
    {
        "id": "fd93a92b685f49ce",
        "type": "inject",
        "z": "84119c5a.51863",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": true,
        "onceDelay": "5",
        "topic": "",
        "payload": "{\"nrchkb\":{\"setup\":{\"name\":\"Office Mood Light\",\"manufacturer\":\"Philips\",\"model\":\"Hue white and color ambiance E26/E27 1600lm\",\"serialNo\":\"1x0000000000010114\",\"firmwareRev\":\"1.88.2\",\"hardwareRev\":\"\",\"softwareRev\":\"20210422\",\"characteristicProperties\":\"{\\\"On\\\":true,\\\"Brightness\\\":true,\\\"Hue\\\":true,\\\"Saturation\\\":true}\"}}}",
        "payloadType": "json",
        "x": 210,
        "y": 480,
        "wires": [
            [
                "7bb7c957f68417a2"
            ]
        ]
    },
    {
        "id": "7bb7c957f68417a2",
        "type": "homekit-service",
        "z": "84119c5a.51863",
        "isParent": true,
        "hostType": "0",
        "bridge": "2a07cbcc.69b214",
        "accessoryId": "",
        "parentService": "",
        "name": "Test",
        "serviceName": "Lightbulb",
        "topic": "",
        "filter": false,
        "manufacturer": "NRCHKB",
        "model": "1.4.3",
        "serialNo": "Default Serial Number",
        "firmwareRev": "1.4.3",
        "hardwareRev": "1.4.3",
        "softwareRev": "1.4.3",
        "cameraConfigVideoProcessor": "ffmpeg",
        "cameraConfigSource": "",
        "cameraConfigStillImageSource": "",
        "cameraConfigMaxStreams": 2,
        "cameraConfigMaxWidth": 1280,
        "cameraConfigMaxHeight": 720,
        "cameraConfigMaxFPS": 10,
        "cameraConfigMaxBitrate": 300,
        "cameraConfigVideoCodec": "libx264",
        "cameraConfigAudioCodec": "libfdk_aac",
        "cameraConfigAudio": false,
        "cameraConfigPacketSize": 1316,
        "cameraConfigVerticalFlip": false,
        "cameraConfigHorizontalFlip": false,
        "cameraConfigMapVideo": "0:0",
        "cameraConfigMapAudio": "0:1",
        "cameraConfigVideoFilter": "scale=1280:720",
        "cameraConfigAdditionalCommandLine": "-tune zerolatency",
        "cameraConfigDebug": false,
        "cameraConfigSnapshotOutput": "disabled",
        "cameraConfigInterfaceName": "",
        "characteristicProperties": "{}",
        "waitForSetupMsg": true,
        "outputs": 2,
        "x": 390,
        "y": 480,
        "wires": [
            [],
            []
        ]
    },
    {
        "id": "e599bc0377f9847a",
        "type": "inject",
        "z": "84119c5a.51863",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": true,
        "onceDelay": "6",
        "topic": "",
        "payload": "{\"On\":true}",
        "payloadType": "json",
        "x": 190,
        "y": 520,
        "wires": [
            [
                "7bb7c957f68417a2"
            ]
        ]
    },
    {
        "id": "175c39cd08337ae8",
        "type": "inject",
        "z": "84119c5a.51863",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": true,
        "onceDelay": "5",
        "topic": "",
        "payload": "{\"nrchkb\":{\"setup\":{\"name\":\"Bedroom Mood Light\",\"manufacturer\":\"Philips\",\"model\":\"Hue white and color ambiance E26/E27 1600lm\",\"serialNo\":\"1x0000000000010115\",\"firmwareRev\":\"1.88.2\",\"hardwareRev\":\"\",\"softwareRev\":\"20210422\",\"characteristicProperties\":\"{\\\"On\\\":true,\\\"Brightness\\\":true,\\\"Hue\\\":true,\\\"Saturation\\\":true}\"}}}",
        "payloadType": "json",
        "x": 250,
        "y": 580,
        "wires": [
            [
                "7485d460095b91e3"
            ]
        ]
    },
    {
        "id": "7485d460095b91e3",
        "type": "homekit-service",
        "z": "84119c5a.51863",
        "isParent": true,
        "hostType": "0",
        "bridge": "2a07cbcc.69b214",
        "accessoryId": "",
        "parentService": "",
        "name": "Test 3",
        "serviceName": "Lightbulb",
        "topic": "",
        "filter": false,
        "manufacturer": "NRCHKB",
        "model": "1.4.3",
        "serialNo": "Default Serial Number",
        "firmwareRev": "1.4.3",
        "hardwareRev": "1.4.3",
        "softwareRev": "1.4.3",
        "cameraConfigVideoProcessor": "ffmpeg",
        "cameraConfigSource": "",
        "cameraConfigStillImageSource": "",
        "cameraConfigMaxStreams": 2,
        "cameraConfigMaxWidth": 1280,
        "cameraConfigMaxHeight": 720,
        "cameraConfigMaxFPS": 10,
        "cameraConfigMaxBitrate": 300,
        "cameraConfigVideoCodec": "libx264",
        "cameraConfigAudioCodec": "libfdk_aac",
        "cameraConfigAudio": false,
        "cameraConfigPacketSize": 1316,
        "cameraConfigVerticalFlip": false,
        "cameraConfigHorizontalFlip": false,
        "cameraConfigMapVideo": "0:0",
        "cameraConfigMapAudio": "0:1",
        "cameraConfigVideoFilter": "scale=1280:720",
        "cameraConfigAdditionalCommandLine": "-tune zerolatency",
        "cameraConfigDebug": false,
        "cameraConfigSnapshotOutput": "disabled",
        "cameraConfigInterfaceName": "",
        "characteristicProperties": "{}",
        "waitForSetupMsg": true,
        "outputs": 2,
        "x": 430,
        "y": 580,
        "wires": [
            [],
            []
        ]
    },
    {
        "id": "0004237a5ac120a4",
        "type": "inject",
        "z": "84119c5a.51863",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": true,
        "onceDelay": "6",
        "topic": "",
        "payload": "{\"On\":true}",
        "payloadType": "json",
        "x": 230,
        "y": 620,
        "wires": [
            [
                "7485d460095b91e3"
            ]
        ]
    },
    {
        "id": "2a07cbcc.69b214",
        "type": "homekit-bridge",
        "bridgeName": "amethyst",
        "pinCode": "198-71-242",
        "port": "",
        "advertiser": "bonjour-hap",
        "allowInsecureRequest": false,
        "manufacturer": "Acheron Systems",
        "model": "ZWHKB",
        "serialNo": "NR1987",
        "firmwareRev": "1.0",
        "hardwareRev": "1.0",
        "softwareRev": "1.0",
        "customMdnsConfig": false,
        "mdnsMulticast": true,
        "mdnsInterface": "",
        "mdnsPort": "",
        "mdnsIp": "",
        "mdnsTtl": "",
        "mdnsLoopback": true,
        "mdnsReuseAddr": true,
        "allowMessagePassthrough": false
    }
]

sjorge commented 2 years ago

I also tested with service2, same result. I'm not 100% sure but I think on restart, for all node with wait for setup it removes them from the bridge, making homekit forget about them, even if they get setup again later with the same info. (perhaps with a different version, but name/serial is always the same)

Shaquu commented 2 years ago

Thanks for example, I will try to look at it soon.

sjorge commented 2 years ago

Were you able to replicate this?

sjorge commented 2 years ago

I think we might be able to close it.

If I use a bridge with only services that have wait for setup enable it works, once you mix in a single services that does not have it enabled it breaks again.

If this is by design (I think it might be) we can close this, although perhaps that should be more clearly documented.

sjorge commented 2 years ago

OK so I figured it out and there may be some room for improvement.

you can't mix wait for setup and normal services on the same bridge
having one service get the setup data a bit later seems to have it end up as a new service*

So the thing seems to be once a service is published, the other services waiting for setup are 'forgotten' and once they do get there setup payload they appear as a new service for homekit. Seems to be a problem with a large amount of services.

The trick is to only use wait for setup service on a the bridge, still some issues if one of the setup payloads gets delayed that service also shows up as a new one.

Some improvements that could be made:

prevent or at least mention it clearly in the docs that you can't mix wait for setup and other services
perhaps add a 'wait time' on the bridge, e.g. 15 seconds --> might allow mixing if all the service with wait for setup complete before that and that should also cover the case where one or two service gets there setup payload a bit delayed.

NRCHKB / node-red-contrib-homekit-bridged