MQTT status update on 'off'

[superjunky]

Hi,

For a while I am noticing a problem when using Siri: On occasion Domoticz-devices are not switched on, apparently because in Homekit they already had status 'on'.

I tested the following scenario:

• When switching 'off' a device in Domoticz,
• next when opening an instance of Homekit, the status remains 'on'
• After closing Homekit and re-opening, the status is updated to 'off'.

I did some thorough testing. This behaviour does not happen while keeping your Homekit instance opened. One way to observe this behaviour is by viewing the accessoires in Homebridge Config UI X:

• When switching 'off' a device in Domoticz,
• The status in Config UI X is updated to 'off'
• next when opening an instance of Homekit, the status in Homekit remains 'on', and in Config UI X the accessoire is switched 'on' again. (The domoticz-status remains 'off' like expected)
• After closing Homekit and re-opening, the status is updated to 'off' in both Homekit and Config UI X.

When using a plugin like MQTTthing for domoticz-devices, this is never the case:

• When opening an instance of Homekit, every domoticz-device added with MQTTthing always display's the updated status.

This behaviour is tested with:

Domoticz: 2020.2 (I am aware of the new 'Prevent Loop' setting for the MQTT hardware plugin) Homebridge: 1.1.0 Edomoticz: 2.1.34 Config UI X: 4.21.0

Config:

{
    "bridge": {
        "name": "Homebridge Test",
        "username": "CC:21:3E:E4:DA:31",
        "port": 51827,
        "pin": "031-45-154"
    },
    "description": "Configuration for Homebridge testing.",
    "platforms": [
        {
            "name": "Config",
            "port": 8087,
            "auth": "form",
            "theme": "cyan",
            "tempUnits": "c",
            "lang": "auto",
            "accessoryControl": {
                "debug": false
            },
            "platform": "config"
        },
        {
            "name": "Domoticz",
            "server": "localhost",
            "port": "8084",
            "roomid": 18,
            "mqtt": {
                "topic": "domoticz/out/mqtt/Homebridge2"
            },
            "ssl": false,
            "dimFix": 0,
            "platform": "eDomoticz"
        }
    ],
    "accessories": [
        {
            "accessory": "mqttthing",
            "type": "valve",
            "valveType": "sprinkler",
            "name": "test-device",
            "topics": {
                "setActive": {
                    "topic": "domoticz/in",
                    "apply": "return JSON.stringify({command: 'switchlight', idx: 6386, switchcmd: message})"
                },
                "getActive": {
                    "topic": "domoticz/out/mqtt/MQTTthing",
                    "apply": "return JSON.parse(message).idx == 6386 ? JSON.parse(message).nvalue > 0 ? 'On' : 'Off' : ''"
                },
                "getInUse": {
                    "topic": "domoticz/out/mqtt/MQTTthing",
                    "apply": "return JSON.parse(message).idx == 6386 ? JSON.parse(message).nvalue > 0 ? 'On' : 'Off' : ''"
                },
                "getStatusActive": {
                    "topic": "domoticz/out/mqtt/MQTTthing",
                    "apply": "return JSON.parse(message).idx == 6386 ? JSON.parse(message).nvalue > 0 ? 'On' : 'Off' : ''"
                }
            },
            "startPub": [
                {
                    "topic": "domoticz/in",
                    "message": "{\"command\": \"getdeviceinfo\", \"idx\": 6386 }"
                }
            ],
            "integerValue": "true",
            "onValue": "On",
            "offValue": "Off",
            "durationTimer": true
        }
    ]
}

[skywalker1979]

i noticed roughly the same behaviour on 2 different installs.....took me some time to understand if it was domoticz, homebridge or homebridge-edomoticz! I also noticed that, at least in my case, not all "devices" are triggering the issue. Like for me it always happens on the garage light...but never with the kitchen light. (at least until now) I can also change the status of the garage light in the homebridge web interface....switch it to off...and after a second or two is becomes on both in the homebridge web interface and homekit. If i change something else, the garage button switches off after a while.

[ludonice54]

Hi ,

Same issue here . In my case problem seems be on switch with extinction delay . In Domoticz i set my switch to turn off after 1sec .

1.When i turn On in homekit my switch turn on and after 1 second turn off . 2.When I turn it On again via Homekit, my switch on homekit stay ON and nothing are send to Domoticz via MQTT . 3.If I turn it off in homekit , request to turn off is sending . 4.If I turn On again i restarted on step 1

homebridge-edomoticz v2.1.34 Homebridge Version 1.1.1

Thanks for your help

[sander1988]

This looks like the same issue I reported some time ago: #193 . I have tested it by disconnecting from the WiFi network at that time, but it also happens occasionally when having the iPhone locked for a long time (it probably suspends some services or WiFi) as you describe.

I have no HomeKit hub (iPad or Apple TV) setup. This was the possible reason for the issue (see #193). But I'm still not 100% sure it is. I would like to know if the issue is also there with a hub setup. If so I try to debug the on/off states to see what's happening.

[superjunky]

I can confirm the issue is there with a hub present.

I have tested every scenario I could think of, and couldn't tie the issue to something obvious. Worth investigation for comparison is the plugin MQTTthing, where this issue does not exists. You can find a accessory-config for a domoticz-dimmer down below.

"accessories": [
        {
            "accessory": "mqttthing",
            "type": "lightbulb",
            "name": "Keuken Spots",
            "topics": {
                "setOn": {
                    "topic": "domoticz/in",
                    "apply": "return JSON.stringify({command: 'switchlight', idx: 468, switchcmd: message})"
                },
                "getOn": {
                    "topic": "domoticz/out/mqtt/MQTTthing",
                    "apply": "return JSON.parse(message).idx == 468 ? JSON.parse(message).nvalue > 0 ? 'On' : 'Off' : ''"
                },
                "setBrightness": {
                    "topic": "domoticz/in",
                    "apply": "return JSON.stringify({command: 'customevent', event: 'hk_keuken_spots', data: message})"
                },
                "getBrightness": {
                    "topic": "domoticz/out/mqtt/MQTTthing",
                    "apply": "return JSON.parse(message).idx == 468 ? JSON.parse(message).svalue1 : ''"
                }
            },
            "startPub": [
                {
                    "topic": "domoticz/in",
                    "message": "{\"command\": \"getdeviceinfo\", \"idx\": 468 }"
                }
            ],
            "integerValue": "true",
            "onValue": "On",
            "offValue": "Off",
            "manufacturer": "FIBARO System",
            "model": "FGD212 Dimmer 2+",
            "serialNumber": "029 (0x1d)"
        }
]

[sander1988]

Ok, thanks. I have added some additional debug output (regarding to the power on/off changes) to the code (in my setup) and started it. I will leave it running the next couple of days to gather some debug logs to see what's going on. Maybe the issue appears earlier so I can check the logs and respond a bit faster.

[skywalker1979]

i also have a apple tv bridge. If it is the same problem, it happens much more often if i keep the homebridge webpage http://localhost:8581 open on the pc and homekit open on the apple device. If you switch something on the webpage it usually happens sooner.

Also i have a switch that stays on only for half second and with a script in domoticz i turn it off after half second it's been turned on. This to me triggers the problem as well. When i press it on homekit on the apple device, 1 time it works being turned off immediately and the next time it does not and stay on.

[superjunky]

Ok, thanks. I have added some additional debug output (regarding to the power on/off changes) to the code (in my setup) and started it. I will leave it running the next couple of days to gather some debug logs to see what's going on. Maybe the issue appears earlier so I can check the logs and respond a bit faster.

Ha yes! Nice to see you picking up the pass on this one :) Let me know if I can be of any assistance

[sander1988]

I think we have a fix. It's currently running in my 'production' environment. I have a way to test it which takes around 15 minutes for each test run. It does now update the on/off state correctly (during two test runs). But let's wait until tomorrow to know for sure the issue is gone here before I create a pull request.

FYI: The 'cachedValues' object gets corrupt due to the characteristic function argument at this line: https://github.com/PatchworkBoy/homebridge-edomoticz/blob/master/lib/domoticz_accessory.js#L150 . This default function value is not passed by object, but it's passed by value. Which means a different key in the 'cachedValues' object.

There also a few other pieces of code where the cache might be corrupt (or in the future). I have fixed these too.

[skywalker1979]

🙏🏻🙏🏻🙏🏻🙏🏻 That sounds great! Thanks

[sander1988]

It seems run stable in my setup. The code is in this repo / branch: https://github.com/sander1988/homebridge-edomoticz/tree/issue210_power_state_fixes

Can someone else also test it before I create a pull request?

Please note: This version includes also another fix (#214) I'm testing currently.

[skywalker1979]

updating file right now....

[skywalker1979]

just tried it....part of the problem seems to be fixed.

Altough i see 2 different problems now: High importance: The devices now are updated correctly, but if i push one of the buttons that gave me the problem before, now on the first press it works, and the second press the icon in homekt is updated but there is no action in domoticz.

Low importance maybe: Some devices have been merged into one. (maybe not important) i will test it for a bit longer.

[skywalker1979]

Actually, after checking better, the behaviour seems to be this: (maybe the problems is different and not related to this issue) 1st click on homekit: Triggers on device. (script on domoticz triggers it immediately off and this works correctly on homekit) 2nd click on homekit - Turn on device.: No trigger on device. - Device stays on in homekit and off in domoticz 3rd click to switch off device in homekit: Only "last updated" time in domoticz is updated. Device remains off.

Device is Zwave...so maybe there is something different going on. Triggering from domoticz results in correct update in domoticz it seems. Anyway, apart from merged devices, it's an improvement. Another thing to say is that i updated only the accessory file. Maybe this fucked up things?

[superjunky]

I have tested the new code:

• The original issue posted here appears solved (happy happy)
• The issue described by skywalker1979 (all great people are born in that year), cannot be reproduced at my setup. Tested with a zwave dimmer.

I'll leave things running and will report back after a few days.

@sander1988 , good stuff!

[sander1988]

Thanks both for testing! Let's wait a couple of days and see if it's stable. Please report back.

Another thing to say is that i updated only the accessory file. Maybe this fucked up things?

You should also update the index.js file. My note: "Please note: This version includes also another fix (#214) I'm testing currently.". This is the pull request from issue #214: https://github.com/PatchworkBoy/homebridge-edomoticz/pull/216/files ; it also changes the index.js .

[skywalker1979]

i missed that part.....i will update asap and see.

[skywalker1979]

I have this script in domoticz:

if      (devicechanged['button'] == 'On'  )
then    
            commandArray['button'] = 'Off'

end

if i trigger the "button" associated after a couple of tries it stops working correctly and the status get out of sync. Is it related? you don't see it? This is also happening with the updated index file

[superjunky]

The status going out of sync is not surprising. Your code is a bit aggressive for Domoticz. I would choose to edit the button and give it a off-delay of 1 sec.

[skywalker1979]

maybe yes...with the off delay at 1 second it is working alright... With the script it's working ok in domoticz. It's only in homebridge where it is not updated correctly.

I also still see randomly some "out of syncs" in the homebridge config webpage and homekit (if i keep open both home in the ios and the homebridge accessory web page and trigger an action from the page) (with no script) maybe this is something else entirely though.

if i open and close homekit (home app on ios.), they both instantly update.

[skywalker1979]

I let it go for a while...it's much improved but i confirm what i wrote above: A button with a off delay of 1 second, (as suggested by superjunky) working on zwave: if i switch it on, after 1 second it correctly switches off. After this point the status is "out of sync." If i click it once more on IOS, there is no response on domoticz and the device does not trigger. If i click it once more again to switch it off on IOS, the "last updated" status on domoticz is updated but nothing else happens. (the relay was already off, the status in domoticz was already off) to ensure there was nothing strange going on, i disabled all "events." doing the same thing on domoticz directly works ok. Anyway the original problem is fixed. I don't know if this is something else...but i never noticed this behaviour before.

[superjunky]

You could try the same with a virtual switch. To make sure its not zwave related

Op 20 aug. 2020 om 15:51 heeft skywalker1979 notifications@github.com het volgende geschreven:

 I let it go for a while...it's much improved but i confirm what i wrote above: A button with a off delay of 1 second, (as suggested by superjunky) working on zwave: if i switch it on, after 1 second it correctly switches off. From this point the status is "out of sync." If i click it once more on IOS, there is no response on domoticz and the device does not trigger. If i click it once more again to switch it off on IOS, the "last updated" status on domoticz is updated but nothing else happens. (the relay was already off, the status in domoticz was already off) to ensure there was nothing strange going on, i disabled all "events." doing the same thing on domoticz directly works ok. Anyway the original problem is fixed. I don't know if this is something else...but i never noticed this behaviour before.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[skywalker1979]

Yes...i did it just now. Same behaviour.

You don't see it? Is it my system screwed? No one else seeing it?

[superjunky]

Yes...i did it just now. Same behaviour.

You don't see it? Is it my system screwed? No one else seeing it?

I use the same, without issues.

[superjunky]

@sander1988 , I have your your code running for 2 days without any side effects.

[sander1988]

@superjunky - Thanks for the update. I have had also running for a few days now without any issues. I have 60 devices in HomeKit and all work fine. I also noticed that the random "device unresponsive" messages I had are gone.

@skywalker1979 - It's not possible me for me to reproduce the issue you have, it seem to be related to a specific setup. But there is one thing I can think of: Your HomeBridge cache might be corrupted. You can try to delete it. The cache file is in your HomeBridge data directory (accessories/cachedAccessories). Stop HomeBridge before deleting it. Note: It might reset some of the devices on your iOS device to their defaults e.g. the room it's in. But most of the time deleting it has no negative side effects.

[skywalker1979]

Actually....i just tried again, with a different raspberry that has only domoticz and homebridge, no zwave. (so really standard) With no updated file, the behaviour is the same. (configure a delay off of one second. From homekit turn on the device. Wait until it switches off and the turn on again. And the switch does not trigger on domoticz.) The wrong behaviour is the same, but is not related to this fix.

[sander1988]

I have created a pull request #219

[skywalker1979]

i guess what i see is this: #206 At least i'm not mad! 🤣

By the way i also see, now that i updated to the latest official versione of the plugin (thus loosing the changes i made with the fix) that if the error reported by superjunky happens, when you switch off the device from homekit, you see the same "Last updated" update. (but nothing else.)

[superjunky]

I have created a pull request #219

Several dimmer-devices are unable to set a level after this fix.

When I edit the level in Homekit, the level jumps back to the initial value. (view attached GIF) I have tested this with a clear cache.

[sander1988]

@superjunky - Did you check again with the previous version of this homebridge-edomoticz to be sure it started after this update? I checked the code (setdValue, getdValue) and don't see any changes here. I have seen the issue too, but I think it started after the last Domoticz upgrade (but I'm not 100% sure). And if is started with this update can you tell me if 'dimFix' is enabled or not?

[skywalker1979]

i confirm i can see the problem too. Dimfix disabled in my setup. If i enable it, it seems to be less reactive while dimming. I can try updating homebridge edomoticz to previous version, but i'm not sure how to do it without screwing the homebridge web interface. Will i screw it if i manually install a version from nodejs command line?

I think something else changed together with this fix, because the other problem i reported got fixed as well. Probably this is a regression from something else. Maybe with homebridge 1.1.2..?

[superjunky]

@sander1988 - Version 2.1.36 does not have this problem. I tested both versions (36 & latest), so it started after this fix. dimFix is set to 0.

My MQTT Explorer shows me for the latest version: double IN messages, and triple OUT messages, upon level change. The previous version sends a single message for each.

It's safe to say we can rule out anything domoticz related.

Please let me know if I can provide you with any additional data.

[sander1988]

Yes there is a message loop while dimming. The iOS device sends a ON command everytime you slide the dimmer on your screen. I have re-added a check here which resolves the issue (in my setup): https://github.com/sander1988/homebridge-edomoticz/commit/5ff6a3fce52b31e3d438df745867cc886f9c928d#diff-ac6ed5cf07ff2f784ccbded0532c1862R153

Now hope that this doesn't bring back the original bug. Let's test it for a couple of days.

Please apply this small fix to your environment manually and let me know if it works.

[simonrb2000]

I can confirm that my dimmers now send an on command each time I move the slider on screen. This causes issues with certain dimmers.

Cheers

On 23 Aug 2020, at 20:33, Sander Peterse notifications@github.com wrote:



Yes there is a message loop while dimming. The iOS device sends a ON command everytime you slide the dimmer on your screen. I have re-added a check here which resolves the issue (in my setup): sander1988@5ff6a3f#diff-ac6ed5cf07ff2f784ccbded0532c1862R153https://github.com/sander1988/homebridge-edomoticz/commit/5ff6a3fce52b31e3d438df745867cc886f9c928d#diff-ac6ed5cf07ff2f784ccbded0532c1862R153

Now hope that this doesn't bring back the original bug. Let's test it for a couple of days.

Please apply this small fix to your environment manually and let me know if it works.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/PatchworkBoy/homebridge-edomoticz/issues/210#issuecomment-678814982, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD2AEAFEQE6Y4KUYJ5N4IALSCFVH7ANCNFSM4NOWL7VQ.

[superjunky]

Great, the check you re-added solves it partially.

• Dimming works solid,
• The original bug is still fixed,
• The only small issue I notice; If switching 'on' a device by setting a level: it results in returning to the last active level before the device was switched 'off'.

This issue makes it impossible to create an automation where a switch is turned on at a certain level. A workaround would be to first turn 'on' the device, and add the new level as a second separate step. It would be nice to dot the i's and cross the t's on this at some point.

Great stuff Sander! Thanks for the fix!

[skywalker1979]

Also #206 seems to stay fixed.

I'm not sure if this is what Superjunky reported, but i only see that if an automation is configured to set a dimming level of for example 15%, when it is activated sometimes it does not reach the set level but stay below. (for example 9%). Not sure if this is what Superjunky reported. At the moment i did not see any problems apart from the above. Thanks for the fix.

[sander1988]

Thanks for the feedback. I have created pull request #220

@superjunky - Can you create a separate issue for the "Switching on by setLevel" problem? Please include the exact steps to reproduce. In which order and how (Domoticz or HomeBridge) do you control the light when turning them on or off. I haven't seen it (yet?) in my setup, so I guess it's related to a specific order.

[superjunky]

@superjunky - Can you create a separate issue for the "Switching on by setLevel" problem? Please include the exact steps to reproduce. In which order and how (Domoticz or HomeBridge) do you control the light when turning them on or off. I haven't seen it (yet?) in my setup, so I guess it's related to a specific order.

Thanks @sander1988 - I've created a new issue https://github.com/PatchworkBoy/homebridge-edomoticz/issues/221