search

paullj1 / TreatLife-HomeKit

Open source firmware for the TreatLife DS0X dimmer switches for native HomeKit use.
MIT License
9 stars 5 forks source link

New HomeKit Architecture #12

Open CaseyTal opened 1 year ago

CaseyTal commented 1 year ago

After upgrading the Apple HomeKit architecture, if the device is power cycled or disconnected from WiFi, it has to be re-added by going to “Add Accessory”, selecting the device and naming it. What really sucks is that I have to remove it and I lose all automations.

Steps to reproduce: Upgrade HomeKit architecture Restart any Home Hub or WiFi.

paullj1 commented 1 year ago

Yeah, I've observed this same behavior when adding someone else to the home, but it doesn't affect all devices, and it doesn't always happen. There have been some changes made to the underlying HomeKit library that have not been merged. Perhaps upgrading the library will solve this.

What exactly do you mean by "Upgrade HomeKit architecture"? I'd love to be able to find a more repeatable way to test this.

On Tue, Nov 15, 2022 at 12:35 AM CaseyTal @.***> wrote:

After upgrading the Apple HomeKit architecture, if the device is power cycled or disconnected from WiFi, it has to be re-added by going to “Add Accessory”, selecting the device and naming it. What really sucks is that I have to remove it and I lose all automations.

Steps to reproduce: Upgrade HomeKit architecture Restart any Home Hub or WiFi.

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSB6PYKSOP24QNXB3LVLWIMOJDANCNFSM6AAAAAASARODQU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

CaseyTal commented 1 year ago

The new HomeKit architecture upgrade is currently in beta, but migrates the entire Apple HomeKit to connect all devices to a hub as a relay all the time instead of just when you’re away. The update is for Matter support, but is supposed to upgrade performance for all devices: https://appleinsider.com/articles/22/11/01/hands-on-with-apples-new-homekit-architecture-in-ios-162

So far, my experience with my treatlife switches with the custom firmware has been that any restart of my WiFi or loss of power makes the switch show “no response” in the HomeKit app. When I hit “+” and then “Add Accessory”, my switches show up in there.

Would it be possible for you to update the firmwares with the latest revisions of dependencies? (Arduino-HomeKit-ESP8266 and WiFi Manager, etc.)?

paullj1 commented 1 year ago

Oh interesting, and also very problematic. Thanks for the explanation!

Upgrading is definitely possible, and something I’ve been meaning to do for a while. Finding the time to do it is another thing. I might have a few minutes to look at it this weekend.

On Fri, Nov 18, 2022 at 06:46 CaseyTal @.***> wrote:

The new HomeKit architecture upgrade is currently in beta, but migrates the entire Apple HomeKit to connect all devices to a hub as a relay all the time instead of just when you’re away. The update is for Matter support, but is supposed to upgrade performance for all devices: https://appleinsider.com/articles/22/11/01/hands-on-with-apples-new-homekit-architecture-in-ios-162

So far, my experience with my treatlife switches with the custom firmware has been that any restart of my WiFi or loss of power makes the switch show “no response” in the HomeKit app. When I hit “+” and then “Add Accessory”, my switches show up in there.

Would it be possible for you to update the firmwares with the latest revisions of dependencies? (Arduino-HomeKit-ESP8266 and WiFi Manager, etc.)?

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12#issuecomment-1319893827, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSB5YKCQWGICADOQXNPDWI5UB3ANCNFSM6AAAAAASARODQU . You are receiving this because you commented.Message ID: @.***>

paullj1 commented 1 year ago

Looking at the commits on the underlying HomeKit library, I don't see anything that would specifically address this. I suspect there's a message that HomeKit passes to each device when the topology changes, or a user is added, and if the device is unreachable (perhaps because it's crashed, and is rebooting) then it doesn't get the message, and does not update its ID.

Looking at the diff between what I have, and what's upstream for the HomeKit lib, the only thing non-documentation thing that's changed is:

--- a/src/crypto.c
+++ b/src/crypto.c
@@ -432,6 +432,13 @@ int crypto_ed25519_sign(
         return -2;
     }

+#if defined(ARDUINO_HOMEKIT_LOWROM)
+    /*  ESP8266 Soft WDT resets with 512k ROMs.
+        Add yield() to perform other tasks and feed WDT.
+    */
+    yield();
+#endif
+

So it won't change anything for the Treatlife chips since they all have 1mb ROMs (at least, some have 2mb). The only other thing I could think to try would be to boot a chip with the debugging compiled in, and watch the output while you do the architecture change. I just pushed a change that will let you more easily get a debug build to see what's going on.

CaseyTal commented 1 year ago

This is what I was referring to with the new architecture: https://support.apple.com/guide/iphone/upgrade-to-the-new-home-architecture-iphaeb0ce2d3/ios

paullj1 commented 1 year ago

Very interesting. I’ll need to pull the new SDK and maybe go fix the third party HomeKit library. Thanks for pointing this out!

On Tue, Dec 13, 2022 at 22:04 CaseyTal @.***> wrote:

This is what I was referring to with the new architecture: https://support.apple.com/guide/iphone/upgrade-to-the-new-home-architecture-iphaeb0ce2d3/ios

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12#issuecomment-1350313865, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSBYOICHGDPTKYLM6G5DWNE2LDANCNFSM6AAAAAASARODQU . You are receiving this because you commented.Message ID: @.***>

paullj1 commented 1 year ago

Have you been able to confirm that this still happens now that 16.2 is out, and the feature is no longer in "beta"? It's looking like the apple developer docs have dropped references to the open source HomeKit Accessory Protocol (HAP) in favor of Matter. I don't see any Matter libraries in Arduino library search, and the official Matter (CHIP) github page only lists support for ESP32 (not ESP8266).

I suspect the next step, is to just drop the homekit library from this project, and implement the Matter spec. That's not going to be very easy or fast, but I'll start working on it.

paullj1 commented 1 year ago

After doing some research on this, it does not appear the ESP8266 will be capable of "commissioning" on a Matter network since it lacks BLE. There may be a way to hack the commissioning part, but then I don't think that HomeKit will accept it. One possible solution, is to use a solder re-work station to remove the existing chip, and replace with something like this. Definitely not ideal, but if they've truly dropped support for the open source HAP SDK, then that may be the only solution going forward.

Not giving up yet, but it's not looking good.

paullj1 commented 1 year ago

Okay, so the HAA firmware (which uses the open source HAP SDK) doesn't appear to struggle with the new architecture, so assuming it's something I'm doing wrong, or something that can be fixed.

paullj1 commented 1 year ago

@CaseyTal, just reluctantly upgraded my architecture, and all of my devices appear to be working fine. I rebooted a few of them, and still connected/registered. Do you think maybe you were having issues because it was in beta when you tried? Leaving the issue open for now, but at this point I am unable to recreate.

CaseyTal commented 1 year ago

For me, I have UniFi WiFi and the devices always show “No Response” after my WiFi system upgrades or a power outage on the router. When that happens, all of my TreatLife switches get set back to pairing mode and have to be removed and re-added and then all automations have to be recreated. For me it happens several times every week.

paullj1 commented 1 year ago

Okay, finally had time to dig into this. It looks like the upstream repo had an issue where it was erasing its device ID during an overly aggressive cleanup routine. I fixed it in a fork, and added my fork as a submodule for building. Pushing v2 shortly which should fix this issue. Sorry for the delay!

CaseyTal commented 1 year ago

No way! I thought it was a lost cause. Every time the power goes out or WiFi upgrades, I cringe knowing that I’m going to have to re-add one or all of the light switches and their automations. Thank you!!!!

CaseyTal commented 1 year ago

Just updated all my lights. Woohoo!!

One quick question: I have DS03 Fan/Light Combo switches. I can change the icon in the home app for the lights, but not the fans. Any way to change that so I can change the fan icon from just the fan blades to a ceiling fan?

if not, no worries. Thank you for working on this!!

paullj1 commented 1 year ago

Woohoo! I’m still in the process of doing all of mine lol

I split them out to show as two different accessories. That lets you change the light logo separately. That’s a function of HomeKit exclusively though. I guess they don’t have an alternate fan logo sadly.

On Sat, Feb 25, 2023 at 3:11 PM CaseyTal @.***> wrote:

Just updated all my lights. Woohoo!!

One quick question: I have DS03 Fan/Light Combo switches. I can change the icon in the home app for the lights, but not the fans. Any way to change that so I can change the fan icon from just the fan blades to a ceiling fan?

if not, no worries. Thank you for working on this!!

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12#issuecomment-1445194724, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSB7GSGNKOH7BFJZMTRLWZJRQDANCNFSM6AAAAAASARODQU . You are receiving this because you commented.Message ID: @.***>

paullj1 commented 1 year ago

Also, you seem to be able to replicate this bug much more consistently than I can. Please let me know if you continue to experience the disconnects. The upstream repo is sort of a mess, and I have some theories about other things that might be causing problems.

On Sat, Feb 25, 2023 at 3:13 PM Paul Jordan @.***> wrote:

Woohoo! I’m still in the process of doing all of mine lol

I split them out to show as two different accessories. That lets you change the light logo separately. That’s a function of HomeKit exclusively though. I guess they don’t have an alternate fan logo sadly.

On Sat, Feb 25, 2023 at 3:11 PM CaseyTal @.***> wrote:

Just updated all my lights. Woohoo!!

One quick question: I have DS03 Fan/Light Combo switches. I can change the icon in the home app for the lights, but not the fans. Any way to change that so I can change the fan icon from just the fan blades to a ceiling fan?

if not, no worries. Thank you for working on this!!

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12#issuecomment-1445194724, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSB7GSGNKOH7BFJZMTRLWZJRQDANCNFSM6AAAAAASARODQU . You are receiving this because you commented.Message ID: @.***>

CaseyTal commented 1 year ago

Haven’t had a single issue with any of the lights disappearing from HomeKit. All has been flawless!!

paullj1 commented 1 year ago

That’s great news, thanks for the update!

On Sat, Mar 11, 2023 at 17:51 CaseyTal @.***> wrote:

Haven’t had a single issue with any of the lights disappearing from HomeKit. All has been flawless!!

— Reply to this email directly, view it on GitHub https://github.com/paullj1/TreatLife-HomeKit/issues/12#issuecomment-1465060885, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPSB2VKX76IQWLPZBDZD3W3UMYNANCNFSM6AAAAAASARODQU . You are receiving this because you commented.Message ID: @.***>

radredgreen commented 4 months ago

I've been having this issue on and off for the last year and was about to give up on my thermostats. So far I'm a week in to this release and no issues. Thank you!!

Paul, I made a small improvement to the thermostat code to accept multiple commands in quick succession more gracefully. This was causing missed commands when setting a scene. I'll send a PR after a little more testing.

radredgreen commented 2 months ago

Hmm, I've gotten 3 more disconnects across 2 thermostats since the last post. Is there a way to dump the flash and regression test the upstream bug?

paullj1 commented 2 months ago

So that was one thing I considered when I had issues updating a few devices (erasing the entire flash before updating). Unfortunately, many of my devices that I had this firmware on just stopped working, and I have been unable to flash the ESP, so I have removed the chip, and gone with an ESP32 + Tasmota install (allowing me to make them Matter devices).

I do have a few functional devices left that I can do testing on, but I won’t really have time to look at it for at least the next month or so. If you can, erasing the firmware, and flashing the latest version could help.