kuestess / homebridge-platform-insteonlocal

Homebridge platform plugin for local Insteon control
Other
76 stars 26 forks source link

Insteon Plugin for HOOBS box locks up the HOOBS box #288

Open greg-e-ctrl opened 1 year ago

greg-e-ctrl commented 1 year ago

Hi,

When I have the Insteon plugin running and no activity on the Insteon network that I can see in the log, it randomly lockups the entire HOOBS box from communicating with the Insteon hub although the log says it did. My Insteon Director app still works correctly when the Hoobs is in this state.

Also, when I use some of my Insteon physical switches or another program that sends a command it logs that it cannot find that accessory ID but in fact it is in the device list and i can control it from Hoobs when all is working correctly. Here is the log from that: 1/13/2023, 8:55:38 AM Platform Insteonlocal BridgeInsteon LocalMessage is from keypad, filtering non-keypad devices. 1/13/2023, 8:55:38 AM Platform Insteonlocal BridgeInsteon LocalFound 0 accessories matching 418C4B 1/13/2023, 8:55:38 AM Platform Insteonlocal BridgeInsteon LocalHub command: {"standard":{"type":"50","id":"418C4B","gatewayId":"3429F8","extended":false,"messageType":1,"hopsLeft":2,"maxHops":3,"command1":"32","command2":"02","raw":"0250418C4B3429F82B3202"}}

I debugged this with Zac from Hoobs and he said to contact you. I would love to get on the phone with you and share my screen with you. I am a software engineer and know something about all this home iot stuff with the restful apis used, etc. :)

Thanks!

-- Greg

kuestess commented 1 year ago

@greg-e-ctrl Looks like the connection to the hub has dropped. Can you share your config? Which version of the plugin are you using?

greg-e-ctrl commented 1 year ago

Hi,

I am using the latest version of everything: HOOBS 4.2.8 Node version: 16.19.0 Model: HSLF-1, System version: 11 Homebridge version: 1.5.0 Paltform InsteonLocal Bridge version: 0.5.7  It is using Bonjour

Attached are my config files.

Can we have a call please?

Thanks!

-- Greg

On 1/14/2023 6:57 AM, kuestess wrote:

@greg-e-ctrl https://github.com/greg-e-ctrl Looks like the connection to the hub has dropped. Can you share your config? Which version of the plugin are you using?

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1382814943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEGDSMDPX432DJBH5B3WSK5FJANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

{ "platform": "InsteonLocal", "name": "InsteonLocal", "port": "25105", "model": "2245", "refresh": "0", "server_port": "3000", "keepAlive": "3600", "host": "192.168.10.35", "user": "admin", "pass": "xxxxx", "devices": [ { "name": "Office Lights", "deviceID": "30D79F", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Living Room Ceiling Fan", "deviceID": "23DED9", "deviceType": "switch", "dimmable": "no" }, { "name": "Garage EV Charger", "deviceID": "418C4B", "deviceType": "outlet", "dimmable": "no", "position": "bottom" }, { "name": "Family Room Lights", "deviceID": "30AF55", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Garage Lights", "deviceID": "43F9BE", "deviceType": "switch", "dimmable": "no" }, { "name": "Garage Lights - Rightside", "deviceID": "43FCDC", "deviceType": "switch", "dimmable": "no" }, { "name": "Dining Room Table Light", "deviceID": "23CB15", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Kitchen Cabinet Lights", "deviceID": "23E076", "deviceType": "switch", "dimmable": "no" }, { "name": "Outside Garage Door Light", "deviceID": "223D58", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Main Bath Sink Lights", "deviceID": "411359", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Main Bath Track Lights", "deviceID": "30379D", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Stairway Lights", "deviceID": "2BD070", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Stairway Lights - Upstairs", "deviceID": "2BCE43", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Kitchen Bar Lights", "deviceID": "3066B1", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Garage Fridge", "deviceID": "2D0CAC", "deviceType": "outlet", "dimmable": "no" }, { "name": "Step Lights", "deviceID": "1B89F8", "deviceType": "switch", "dimmable": "no" }, { "name": "Step Lights - Upper", "deviceID": "1B84B8", "deviceType": "switch", "dimmable": "no" }, { "name": "Inside Foyer Light - Garage", "deviceID": "2BCC82", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Inside Foyer Light", "deviceID": "303BBA", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Garage Bench Light", "deviceID": "213E51", "deviceType": "outlet", "dimmable": "no" }, { "name": "Kitchen Table Light", "deviceID": "23CA25", "deviceType": "dimmer", "dimmable": "yes" }, { "name": "Kitchen Sink Lights", "deviceID": "30B5FC", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Hot Water Heater Leak Sensor", "deviceID": "550270", "deviceType": "leaksensor" }, { "name": "Family Room Table Light", "deviceID": "364712", "dimmable": "no", "deviceType": "outlet" }, { "name": "Hot Water Pump", "deviceID": "549E5D", "dimmable": "no", "deviceType": "outlet" }, { "name": "Instant Hot Water", "deviceID": "549E1B", "dimmable": "no", "deviceType": "outlet" }, { "name": "Family Room Leak Sensor", "deviceID": "3F9F96", "dimmable": "no", "deviceType": "leaksensor" }, { "name": "Mailbox Door", "deviceID": "36DC97", "dimmable": "no", "deviceType": "doorsensor" }, { "name": "Garage�Side�Door", "deviceID": "37DFD8", "dimmable": "no", "deviceType": "doorsensor" }, { "name": "Garage Leak Sensor", "deviceID": "377CF1", "dimmable": "no", "deviceType": "leaksensor" }, { "name": "Kitchen�Sink Leak�Sensor - Left Front", "deviceID": "47B2C0", "dimmable": "no", "deviceType": "leaksensor" }, { "name": "Kitchen�Sink Leak�Sensor - Right Front", "deviceID": "55057D", "dimmable": "no", "deviceType": "leaksensor" }, { "name": "Kitchen�Sink Leak�Sensor - Right Back", "deviceID": "55048C", "dimmable": "no", "deviceType": "leaksensor" }, { "name": "Dining Room Lights", "deviceID": "303E0B", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Living Room Lights", "deviceID": "3042D4", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Kitchen Lights", "deviceID": "2C0270", "dimmable": "yes", "deviceType": "keypad", "keypadbtn": "ON" }, { "name": "Crawl Space Door Sensor", "deviceID": "47FBEE", "dimmable": "no", "deviceType": "doorsensor" }, { "name": "Holiday Wreath", "deviceID": "2D1418", "dimmable": "no", "deviceType": "outlet" }, { "name": "Fish Tank Light", "deviceID": "510F24", "dimmable": "no", "deviceType": "outlet", "position": "top" }, { "name": "Upstairs Lights", "deviceID": "2C01F8", "dimmable": "yes", "deviceType": "keypad", "keypadbtn": "ON" }, { "name": "Upstairs Lights - Laundry", "deviceID": "304456", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Outside Foyer Lights", "deviceID": "24AC51", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Main�Bedroom�Reading�Lights", "deviceID": "30ADE8", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Main�Bedroom�Ceiling�Lights", "deviceID": "30A703", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Kitchen Lights - Bar", "deviceID": "3066DA", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Hallway Light - Back", "deviceID": "309978", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Hallway Light", "deviceID": "30BD0E", "dimmable": "yes", "deviceType": "dimmer" }, { "name": "Sideyard Lights", "deviceID": "542579", "dimmable": "no", "deviceType": "switch" } ] }

kuestess commented 1 year ago

@greg-e-ctrl Do you get any messages in your console that say 'Started connection watcher...' and 'Reconnecting to Hub...'? From your config, you should see these about every hour. The 2245 gets 'bored' with the connection sometimes, so the plugin periodically disconnects/reconnects the hub. Sorry, don't do phone support but happy to help here.

AstonMartin715 commented 1 year ago

I have the exact same issue, with the same setup, as the OP. I rebooted my hoops hub at 3:11 PM. It worked until 3:42 PM, which is when it sent a CLOSE and RECONNECT command to the Insteon hub (I have my keep alive set to 1800). After it sent the RECONNECT command it gave the ETIMEDOUT error and no longer speaks to the Insteon hub.

greg-e-ctrl commented 1 year ago

Yep, you are seeing same: i left Keepalive at default of 3600 and even tried it at 120 (every tweo mins) and it still locks up eventually within a few hours.

1/14/2023, 10:43:47 AMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 10:43:52 AMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 10:43:52 AMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub...1/14/2023, 11:43:47 AMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 11:43:52 AMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 11:43:52 AMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub...1/14/2023, 12:43:47 PMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 12:43:52 PMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 12:43:52 PMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub...

e-ctrl

On 1/14/2023 3:50 PM, MarcStapley wrote:

I have the exact same issue, with the same setup, as the OP. I rebooted my hoops hub at 3:11 PM. It worked until 3:42 PM, which is when it sent a CLOSE and RECONNECT command to the Insteon hub (I have my keep alive set to 1800). After it sent the RECONNECT command it gave the ETIMEDOUT error and no longer speaks to the Insteon hub.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1382966488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEGCTKOX6IBYADZTEQLWSM3S3ANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

greg-e-ctrl commented 1 year ago

Hi Folks,

Okay, the Hoobs box just locked up again and even though it says it is turning my lights on and off, they are not responding.

It started at 8:46:01 this morning: 1/15/2023, 8:12:06 AMPlatform Insteonlocal BridgeInsteonLocalGot off event for Kitchen Cabinet Lights 1/15/2023, 8:43:47 AMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub... 1/15/2023, 8:43:52 AMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub... 1/15/2023, 8:45:39 AMPlatform Insteonlocal BridgeInsteonLocalSetting power state of Step Lights to on 1/15/2023, 8:45:50 AMPlatform Insteonlocal BridgeInsteonLocalError setting power state of Step Lights to on 1/15/2023, 8:45:50 AMPlatform Insteonlocal BridgeInsteonLocalGetting status for Step Lights 1/15/2023, 8:46:00 AMPlatform Insteonlocal BridgeInsteonLocalError getting power state of Step Lights 1/15/2023, 8:46:00 AMPlatform Insteonlocal BridgeInsteonLocalSetting power state of Step Lights to off 1/15/2023, 8:46:01 AMPlatform Insteonlocal BridgeWARNINGunhandled rejection: Error: connect ETIMEDOUT 192.168.10.35:25105 1/15/2023, 8:46:10 AMPlatform Insteonlocal BridgeInsteonLocalSetting power state of Step Lights - Upper to on 1/15/2023, 8:46:10 AMPlatform Insteonlocal BridgeInsteonLocalError setting power state of Step Lights to off 1/15/2023, 8:46:10 AMPlatform Insteonlocal BridgeInsteonLocalGetting status for Step Lights 1/15/2023, 8:46:18 AMPlatform Insteonlocal BridgeInsteonLocalSetting power state of Step Lights - Upper to off 1/15/2023, 8:46:20 AMPlatform Insteonlocal BridgeInsteonLocalError setting power state of Step Lights - Upper to on 1/15/2023, 8:46:20 AMPlatform Insteonlocal BridgeInsteonLocalGetting status for Step Lights

When locked up like this: 1) The Hoobs GUI app does not control the devices even though it thinks it is 2) HomeKit and Siri think they are work but they also do not work 3) The Insteon Director app and the OG Insteon app continue to work correctly! 4) The only fix is to reboot the HOOBS box, restarting the Insteon plugin/bridge does NOT fix this problem.

Therefore, it appears to be a Hoobs box networking issue!

e-ctrl

On 1/14/2023 5:08 PM, e-ctrl wrote:

Yep, you are seeing same: i left Keepalive at default of 3600 and even tried it at 120 (every tweo mins) and it still locks up eventually within a few hours.

1/14/2023, 10:43:47 AMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 10:43:52 AMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 10:43:52 AMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub...1/14/2023, 11:43:47 AMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 11:43:52 AMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 11:43:52 AMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub...1/14/2023, 12:43:47 PMPlatform Insteonlocal BridgeInsteonLocalClosing connection to Hub...1/14/2023, 12:43:52 PMPlatform Insteonlocal BridgeInsteonLocalReconnecting to Hub...1/14/2023, 12:43:52 PMPlatform Insteonlocal BridgeInsteonLocalConnected to Insteon Model 2245 Hub... e-ctrl On 1/14/2023 3:50 PM, MarcStapley wrote:

I have the exact same issue, with the same setup, as the OP. I rebooted my hoops hub at 3:11 PM. It worked until 3:42 PM, which is when it sent a CLOSE and RECONNECT command to the Insteon hub (I have my keep alive set to 1800). After it sent the RECONNECT command it gave the ETIMEDOUT error and no longer speaks to the Insteon hub.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1382966488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEGCTKOX6IBYADZTEQLWSM3S3ANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

kuestess commented 1 year ago

@greg-e-ctrl Agree that this is a Hoobs-specific (likely) networking issue. I spun up a Hoobs instance this morning and it's been running fine for me for a bit, but I don't have much Insteon network traffic on that instance. Did this just start after the latest update to 0.5.7? If so, you might try setting the keepAlive to 0 (disabled) to see if that helps.

AstonMartin715 commented 1 year ago

Did some extensive testing today. Exact same issue as above. Communication between the hoobs and Insteon works fine until someone presses an Insteon switch manually, then this happens for every device, after which the communication no longer works ... 1/15/2023, 10:50:40 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] Chandelier (MB) is off at 0% 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] Received command for 3CE278 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] Found 1 accessories matching 3CE278 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] Hub command: {"standard":{"type":"50","id":"3CE278","gatewayId":"000001","extended":false,"messageType":6,"hopsLeft":2,"maxHops":3,"command1":"11","command2":"00","raw":"02503CE278000001CB1100"}} 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] Got event for Chandelier (MB) (3CE278) 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal Got on event for Chandelier (MB) 1/15/2023, 10:50:58 AM Platform Insteonlocal Bridge 2 InsteonLocal [ DEBUG ] No group members defined for Chandelier (MB)

kuestess commented 1 year ago

@AstonMartin715 Did this just start after the latest update to 0.5.7? If so, you might try setting the keepAlive to 0 (disabled) to see if that helps.

AstonMartin715 commented 1 year ago

No I've been dealing with this for about 6 months and through multiple updates. I have set the keep alive to 0 as you suggested and am re-testing. Since doing that I've had 2 hours of consistent uptime.

greg-e-ctrl commented 1 year ago

Hi,

Glad to see @AstonMartin715 is seeing the same as I am so this is reproducible!

Also the same is that this has been happening for me for the past 6+ months as well since I first purchased the Hoobs box. So it existed prior to the 0.5.7 release for sure.

I too just set the KeepAlive to 0 so now both Refresh and Keep Alive are both set to 0 and let's see what happens.

Question: Does anyone know if the Insteon hub has some sort of API call rate limit if it is from a non-native/non-Insteon controller/3rd-party app sending it commands?

I'll report back after letting this new keep alive setting bakes in... I have to first make the change in the Insteon bridge setting to the Keep Alive and then go into the Hub Settings and Reboot the entire Hoobs box to get it working again. :( argh

Thank all for you help in debugging this! We will find it eventually!!! :)

-- Greg

On 1/15/2023 12:45 PM, AstonMartin715 wrote:

No I've been dealing with this for about 6 months and through multiple updates. I have set the keep alive to 0 as you suggested and am re-testing. Since doing that I've had 2 hours of consistent uptime.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1383248699, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEENHVLYZL7NMRBKCLLWSROWJANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

greg-e-ctrl commented 1 year ago

Good Morning folks!

So it locked up again last night after responding correctly for at least 4 hours before i went to bed.

But when I tried first thing this morning, no-go. Since no Keep Alive or Refresh pings are happening, i did not see any ETIMEOUT warnings in the log; actually nothing at all in the log from the last time i turned a light off last night before bed. Until this morning when i used the HOOBS GUI app on Windows to try to turn on a light that did not work and again i get the ETIMEOUT warnings for that request...

So again, this tells me that it is not the Insteon plugin per se, that is causing the issue, but instead appears to be the hoobs box network interface "going to sleep" whereas requests are going into the ether/bit bucket unanswered? And recall this is the state it is in when this happens: 1) The Hoobs GUI app does not control the devices even though it thinks it does and marks them completed in the log 2) HomeKit and Siri also think they are working but they do not; again traced in the log 3) The Insteon Director app and the OG Insteon app continue to work correctly! 4) The only fix is to reboot the HOOBS box, restarting the Insteon plugin/bridge does NOT fix this problem.

Another observation: It appears there may be a memory leak. After a fresh reboot, memory usage hovers around 41MB used and after running all night and after it is in the lockeup state it is using around 54MB.

Thanks folks and happy MLKjr Day!

On 1/15/2023 2:26 PM, e-ctrl wrote:

Hi,

Glad to see @AstonMartin715 is seeing the same as I am so this is reproducible!

Also the same is that this has been happening for me for the past 6+ months as well since I first purchased the Hoobs box. So it existed prior to the 0.5.7 release for sure.

I too just set the KeepAlive to 0 so now both Refresh and Keep Alive are both set to 0 and let's see what happens.

Question: Does anyone know if the Insteon hub has some sort of API call rate limit if it is from a non-native/non-Insteon controller/3rd-party app sending it commands?

I'll report back after letting this new keep alive setting bakes in... I have to first make the change in the Insteon bridge setting to the Keep Alive and then go into the Hub Settings and Reboot the entire Hoobs box to get it working again. :( argh

Thank all for you help in debugging this! We will find it eventually!!! :)

-- Greg

On 1/15/2023 12:45 PM, AstonMartin715 wrote:

No I've been dealing with this for about 6 months and through multiple updates. I have set the keep alive to 0 as you suggested and am re-testing. Since doing that I've had 2 hours of consistent uptime.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1383248699, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEENHVLYZL7NMRBKCLLWSROWJANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

kuestess commented 1 year ago

@greg-e-ctrl Agree with your diagnosis - everything you describe points to a network issue specific to Hoobs. The apps will continue to work as they initiate a fresh connection to the hub, which the plugin should do as well...but seems it isn't making it out of the Hoobs box. Quick thought - have you tried power cycling your Hub or spinning up a quick Homebridge container to see it it persists?

greg-e-ctrl commented 1 year ago

Hi @kuestess,

Do you mean to power cycle my Hoobs box or Insteon Hub?

And how do I "spin up" a Homebridge container? I already created one using the Homemanager App on my iPhone when i first set this up; unless you are referring to something else?

Thank you!

On 1/16/2023 8:11 AM, kuestess wrote:

@greg-e-ctrl https://github.com/greg-e-ctrl Agree with your diagnosis - everything you describe points to a network issue specific to Hoobs. The apps will continue to work as they initiate a fresh connection to the hub, which the plugin should do as well...but seems it isn't making it out of the Hoobs box. Quick thought - have you tried power cycling your Hub or spinning up a quick Homebridge container to see it it persists?

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1384265248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEDDUCCAN4UYX46FEJTWSVXLBANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

kuestess commented 1 year ago

@greg-e-ctrl I meant to power cycle your Hub. Sometimes they get a bit wonky, and unfortunately no way to reboot them without unplugging. 'Spin up' is nerd speak :-) for setting up a docker container to run/administrate home bridge. See here: https://github.com/oznu/homebridge-config-ui-x

AstonMartin715 commented 1 year ago

Confirming your 4 points above are exactly the same for me @greg-e-ctrl. Tested a lot today. Set up an iPhone shortcut to cycle one light every 10 minutes. It failed as soon as someone manually turned on a different light using the Insteon switch which caused the Hoobs hub to cycle through all of the devices, saying it couldn’t find them:l, for example:

1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalReceived command for 3C68E0 1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalMessage is from keypad, filtering non-keypad devices. 1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalFound 0 accessories matching 3C68E0

After that there was no communication between the Hoobs hub and the Insteon hub. The only error I get is:

1/16/2023, 7:44:43 PMPlatform Insteonlocal Bridge 2InsteonLocalError getting power state of Garage Lights (G)

It seems a call from the Insteon hub back to the Hoobs hub (instead of the other way round) throws it through a bit of a loop.

greg-e-ctrl commented 1 year ago

ok, yes @AstonMartin715, we are seeing the same issues...

BTW, I just rebooted EVERYTHING and it still locked up in a couple hours, without any other Insteon devices turned/on/off or activity whatsoever during that time (we left the house) and it still locked up.

Does this have anything to do with how connected to the network? Via hard wired ethernet cable or via WiFi?

If you can sniff the network (wireshark?) to see if in fact the commands are comming out of the HOOBS box destined for the Insteon HUB that would also tell us something...

Thanks!

On 1/16/2023 11:54 AM, AstonMartin715 wrote:

Confirming your 4 points above are exactly the same for me @greg-e-ctrl https://github.com/greg-e-ctrl. Tested a lot today. Set up an iPhone shortcut to cycle one light every 10 minutes. It failed as soon as someone manually turned on a different light using the Insteon switch which caused the Hoobs hub to cycle through all of the devices, saying it couldn’t find them:l, for example:

1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalReceived command for 3C68E0 1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalMessage is from keypad, filtering non-keypad devices. 1/16/2023, 7:26:42 PMPlatform Insteonlocal Bridge 2InsteonLocalFound 0 accessories matching 3C68E0

After that there was no communication between the Hoobs hub and the Insteon hub. The only error I get is:

1/16/2023, 7:44:43 PMPlatform Insteonlocal Bridge 2InsteonLocalError getting power state of Garage Lights (G)

It seems a call from the Insteon hub back to the Hoobs hub (instead of the other way round) throws it through a bit of a loop.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1384490248, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEE53SFMYF7XBLLK6DTWSWRNXANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

AstonMartin715 commented 1 year ago

I have tried ethernet only, wi-fi only and both at the same time. I have discovered that even when the Hoobs hub loses connection to the Insteon hub, the Insteon will still take instructions from the Amazon Echo. I assume the communication protocol is the same for both devices? So does that tell us it's not so much Insteon not accepting commands, rather Hoobs not actually sending (or receiving) them?

greg-e-ctrl commented 1 year ago

Sounds correct to me! That the HOOBS box is not sending the commands to the Insteon hub. Can we please get the HOOBS folks to chime in on this thread? @Zach, are you seeing this?--GregOn Jan 16, 2023, at 7:34 PM, AstonMartin715 @.***> wrote: I have tried ethernet only, wi-fi only and both at the same time. I have discovered that even when the Hoobs hub loses connection to the Insteon hub, the Insteon will still take instructions from the Amazon Echo. I assume the communication protocol is the same for both devices? So does that tell us it's not so much Insteon not accepting commands, rather Hoobs not actually sending them?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

AstonMartin715 commented 1 year ago

I tried to turn on verbose logging on Hoobs so I could see what was going on but I couldn’t get it to work (sudo hoobsd hub -v is the command I got from the Hoobs online CLI manual). Anyone else have any luck with this? I was able to turn on debugging mode but that didn’t tell me anything new.

zzack01 commented 1 year ago

@kuestess please send me an email

kuestess commented 1 year ago

@AstonMartin715 @greg-e-ctrl Are either of you using any software to connect to your Hub in addition to the plugin (something like Home Assistant that would maintain a connection)?

AstonMartin715 commented 1 year ago

I am not. Only the Hoobs hub. I added the Amazon Echo skill for Insteon a few days ago to test that the Insteon hub could still receive commands from a 3rd party device when it was not receiving command from Insteon (see above).

kuestess commented 1 year ago

Thanks @AstonMartin715 - working with @zzack01 and just trying to eliminate things that could be grabbing the connection to the Hub.

AstonMartin715 commented 1 year ago

Is there an easy way to make the Insteon hub a local LAN device only and stop it from receiving any pings from the WAN? I would be OK with that as I have no reason to access it from outside of the LAN if the Apple Homekit/Hoobs/Insteon bridge is working as it should. I did some research and just have a gut feeling that making Insteon a local only device might help. The other option might be to put Hoobs and Insteon both on their own network (eg. 192.168.5.xxx) and firewall that off from other traffic except Homekit?

greg-e-ctrl commented 1 year ago

I have a HomeKit Bridge for using my iPhone Home app. It uses the Homemanager app to activate.  I also have the orig Insteon app and the new Insteon Director app as well. Home app works well until the HOOBS box locks up. Both Insteon apps continue to work perfectly ALL THE TIME even when HOOBS box is locked up. I didn’t think any of the above apps maintain a persistent connection to hub. I thought they all just use stateless  RESTful API calls?Suggestion: use Wireshark or other network sniffer snd see how the new Insteon Director app talks to the Insteon Hub and then just mimic that in the HOOBS box using same protocol/ handshaking as you see in the sniffer packets from the Director app. Thanks!--GregOn Jan 20, 2023, at 12:08 PM, kuestess @.***> wrote: @AstonMartin715 @greg-e-ctrl Are either of you using any software to connect to your Hub in addition to the plugin (something like Home Assistant that would maintain a connection)?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

AstonMartin715 commented 1 year ago

I put the Hoobs hub and Insteon hub on their own separate VLAN. No other devices present. They talked to each other for a couple of hours and then the same problem as described above occurred.

kuestess commented 1 year ago

@zzack01 Any updates?

greg-e-ctrl commented 1 year ago

Hi @kuestess and @zach,

Yes, What is the status here?

I have seen 3-4 updates released and applied them but nothing has changed wrt to our issue.

Also, now when i check for updates, i get the following:

2/2/2023, 7:43:23 AM deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x bullseye main 2/2/2023, 7:43:23 AM deb-src [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x bullseye main 2/2/2023, 7:43:24 AM Hit:1 http://deb.debian.org/debian bullseye InRelease 2/2/2023, 7:43:24 AM Hit:2 http://deb.debian.org/debian bullseye-updates InRelease 2/2/2023, 7:43:24 AM Hit:3 http://deb.debian.org/debian bullseye-backports InRelease 2/2/2023, 7:43:24 AM Hit:4 http://security.debian.org bullseye-security InRelease 2/2/2023, 7:43:25 AM Hit:6 https://deb.nodesource.com/node_16.x bullseye InRelease 2/2/2023, 7:43:25 AM Hit:7 https://dl.hoobs.org/debian stable InRelease 2/2/2023, 7:43:25 AM Get:8 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB] 2/2/2023, 7:43:25 AM Hit:5 http://mirrors.jevincanders.net/armbian/apt bullseye InRelease 2/2/2023, 7:44:10 AM Err:8 https://dl.yarnpkg.com/debian stable InRelease 2/2/2023, 7:44:10 AM The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.> 2/2/2023, 7:44:10 AM Fetched 17.1 kB in 6s (2,908 B/s) 2/2/2023, 7:44:10 AM Reading package lists... 2/2/2023, 7:44:11 AM W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.> 2/2/2023, 7:44:11 AM W: Failed to fetch https://dl.yarnpkg.com/debian/dists/stable/InRelease  The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.***> 2/2/2023, 7:44:11 AM W: Some index files failed to download. They have been ignored, or old ones used instead.

Is there anything i need to do to correct this?

Thanks!

-- Greg

On 2/1/2023 4:52 PM, kuestess wrote:

@zzack01 https://github.com/zzack01 Any updates?

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1412990087, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEHPSGUIKJGEKQXBTSLWVMAN5ANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

greg-e-ctrl commented 1 year ago

Hi Folks,

Are we being ghosted here? :)

We would appreciate an update after a week with no status...

Thanks!

-- Greg

On 2/2/2023 7:51 AM, e-ctrl wrote:

Hi @kuestess and @zach,

Yes, What is the status here?

I have seen 3-4 updates released and applied them but nothing has changed wrt to our issue.

Also, now when i check for updates, i get the following:

2/2/2023, 7:43:23 AM deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x bullseye main 2/2/2023, 7:43:23 AM deb-src [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x bullseye main 2/2/2023, 7:43:24 AM Hit:1 http://deb.debian.org/debian bullseye InRelease 2/2/2023, 7:43:24 AM Hit:2 http://deb.debian.org/debian bullseye-updates InRelease 2/2/2023, 7:43:24 AM Hit:3 http://deb.debian.org/debian bullseye-backports InRelease 2/2/2023, 7:43:24 AM Hit:4 http://security.debian.org bullseye-security InRelease 2/2/2023, 7:43:25 AM Hit:6 https://deb.nodesource.com/node_16.x bullseye InRelease 2/2/2023, 7:43:25 AM Hit:7 https://dl.hoobs.org/debian stable InRelease 2/2/2023, 7:43:25 AM Get:8 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB] 2/2/2023, 7:43:25 AM Hit:5 http://mirrors.jevincanders.net/armbian/apt bullseye InRelease 2/2/2023, 7:44:10 AM Err:8 https://dl.yarnpkg.com/debian stable InRelease 2/2/2023, 7:44:10 AM The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.> 2/2/2023, 7:44:10 AM Fetched 17.1 kB in 6s (2,908 B/s) 2/2/2023, 7:44:10 AM Reading package lists... 2/2/2023, 7:44:11 AM W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.> 2/2/2023, 7:44:11 AM W: Failed to fetch https://dl.yarnpkg.com/debian/dists/stable/InRelease The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging @.***> 2/2/2023, 7:44:11 AM W: Some index files failed to download. They have been ignored, or old ones used instead.

Is there anything i need to do to correct this?

Thanks!

-- Greg

On 2/1/2023 4:52 PM, kuestess wrote:

@zzack01 https://github.com/zzack01 Any updates?

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1412990087, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEHPSGUIKJGEKQXBTSLWVMAN5ANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

kuestess commented 1 year ago

@greg-e-ctrl Happy to support the plugin, but I can't address issues (likely) with Hoobs. I created an issue on the Hoobs repo (referenced above) as it will likely get more attention there.

greg-e-ctrl commented 1 year ago

Totally understand @kuestess

TY and appreciate it!

-- Greg

On 2/9/2023 4:39 PM, kuestess wrote:

@greg-e-ctrl https://github.com/greg-e-ctrl Happy to support the plugin, but I can't address issues (likely) with Hoobs. I created an issue on the Hoobs repo (referenced above) as it will likely get more attention there.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1425021313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEFAW6MHAEJYJ52HDRLWWWE5DANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

zzack01 commented 1 year ago

@greg-e-ctrl & @AstonMartin715

zzack01 commented 1 year ago

for the yarn error, please open the terminal and run sudo yarn cache clean

Can you also try to do a reflash on your HOOBS™ box and have a new install of the insteon plugin to see if your issue continues? https://support.hoobs.org/topics/resetting-re-flashing/how-to-i-re-flash-the-software

For testing lets only add a new install of the insteon plugin Also if you are using insteon for anything else lets break that bridge for now and just run the insteon plugin on the insteon system.

### ETIMEDOUT issue seems to be with the plugin as non HOOBS™ users are having/ have had it too: https://github.com/kuestess/homebridge-platform-insteonlocal/issues/11#issuecomment-396105247 https://github.com/kuestess/homebridge-platform-insteonlocal/issues/166#issuecomment-1017615324 https://github.com/kuestess/homebridge-platform-insteonlocal/issues/166#issuecomment-1086649035 https://github.com/kuestess/homebridge-platform-insteonlocal/issues/166#issuecomment-1153207034 https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1382966488 https://github.com/kuestess/homebridge-platform-insteonlocal/issues/281#issue-1368654043

greg-e-ctrl commented 1 year ago

Zach,

Dude! I'm sorry, but...

As per your instructions earlier, this is exactly what I have done.

And I have applied all updates; always do.

I reboot using the HOOBS GUI interface only.

There have been no power outages as I have solar and Powerwall batteries on my house.

So unless there has been a fix SPECIFICALLY for this issue released by the HOOBS group, then this issue still exists and a fresh flashed uSD card will not solve the problem!

Am I missing something?

Thanks!

-- greg

On 2/10/2023 10:10 AM, zzack01 wrote:

Can you try to do a reflash on your HOOBS™ box and have a new install of the insteon plugin to see if your issue continues? https://support.hoobs.org/topics/resetting-re-flashing/how-to-i-re-flash-the-software

For testing lets only add the insteon plugin, I too am running insteon and do not have the issue you are having...

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1426164221, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEDTOTEICMM6GIQ4IQ3WW2ABPANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

AstonMartin715 commented 1 year ago

I have reflashed the Hoobs box @zzack01, installed the latest update (4.2.8) , and installed the Insteon plugin. I have no other bridges to Insteon. The service started at 8:52 PST. I'll periodically test it and let you know what happens.

AstonMartin715 commented 1 year ago

Worked for about 12 hours and then just stopped working entirely. Between when I know that it last worked and when it didn’t (less than an hour) nothing happened to the network, either box, power, etc. And nothing showing up in the log.

greg-e-ctrl commented 1 year ago

and mine has crashed twice since... so no change in behavior

On 2/12/2023 10:15 AM, AstonMartin715 wrote:

Worked for about 12 hours and then just stopped working entirely. Between when I know that it last worked and when it didn’t (less than an hour) nothing happened to the network, either box, power, etc. And nothing showing up in the log.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1427097617, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUECYY3UKZ3MPD4NNZCLWXESFDANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

AstonMartin715 commented 1 year ago

Not sure if this is helpful but I've noticed that the connection from the Hoobs hub to the Insteon hub re-engages now and then without re-booting Hoobs, including sending a command hours later. I can't believe Insteon has capacity to buffer and not execute a command, so I'm guessing it's the Hoobs hub/Bridge that is delayed in sending the command? Which reinforces that something is happening on the Hoobs/bridge end to lose the connection to Insteon.

AstonMartin715 commented 1 year ago

Couple of other updates. The Insteon bridge definitely fails after someone presses a manual Insteon switch. I know @greg-e-ctrl previously confirmed his failed without that taking place, mine may do so too. But I have been able to force a failure this way. And today I added a Meross bridge to test it out. When the Insteon bridge fails, the Meross one still works, so the Hoobs hub is online and sending signals, just not to Insteon.

zzack01 commented 1 year ago

@kuestess do you think that there could be an issue with the insteon hub? Would a reset of the insteon hub and resetup help? I have been using the insteon plugin since Insteon closed and do not have this issue, I have reached out to a few others that have helped in the past and they are not having any issues. So maybe there is some thing wrong with the insteon hub?

AstonMartin715 commented 1 year ago

@zzack01 since a reboot of the Hoobs hub always solves the problem, at least for a while, that makes me suspicious that it would be an Insteon hub issue. Rebooting the Insteon hub does not solve it ever until the Hoobs hub is rebooted. Given everything I've seen, this feels like it's on the Hoobs/Plugin side of the equation. May be clutching at straws here but curious if there's a way to write a script so that the Hoobs hub sends a command to Insteon every 5 minutes to switch on or off a single source, and can then detect the status of that source back from Insteon, so we can identify more closely when the failure occurs and what might be causing it? Absent that I'm running out of ideas for diagnosis or process of elimination.

zzack01 commented 1 year ago

That would be a great question for @kuestess, if you were to do a reflash on your HOOBS™ hub and ran another plugin like hue would you still have the same issue where you need to reboot the HOOBS™ box, Also by rebooting the HOOBS™ box without a proper shut down you run the risk of a "corruption".

AstonMartin715 commented 1 year ago

I did the reflash (see above) and ran with just the Insteon plugin for a while. Same problem. I then added the Meross plugin, and it works fine without a reboot. What is the correct way to reboot with a proper shut down? I use the reboot button under the hub settings. Should I be doing something else?

zzack01 commented 1 year ago

That is the correct way, a lot of users just unplug and replug, so other plugins run fine on HOOBS™, it is only the insteon plugin?

greg-e-ctrl commented 1 year ago

Hi,

When I manually press an Insteon physical switch it does NOT lockup my HOOBS box as @AstonMartin715 experiences. I can still control other Insteon devices via the HOOBS box after I have physically used other Insteon switches.

--

e-ctrl

On 2/15/2023 10:02 AM, AstonMartin715 wrote:

Couple of other updates. The Insteon bridge definitely fails after someone presses a manual Insteon switch. I know @greg-e-ctrl https://github.com/greg-e-ctrl previously confirmed his failed without that taking place, mine may do so too. But I have been able to force a failure this way. And today I added a Meross bridge to test it out. When the Insteon bridge fails, the Meross one still works, so the Hoobs hub is online and sending signals, just not to Insteon.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1431787102, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEE5CJQ6OERAU3PB55DWXUK2ZANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

greg-e-ctrl commented 1 year ago

Correct, it is NOT an Insteon Hub issue. The Insteon Hubs are rock solid and have been for many years. Rebooting it does not change the behavior of this issue. When the HOOBS box locks up the Insteon apps can still control all the Insteon devices using the Insteon Hub; so the issue is definitely in the HOOBS box!

However, I did just notice that a new version 0.5.8 is available for the Insteon Plugin. @kuestess, I see in Github you fixed the following:

"Fixes for deleting links and missing devices array"

Does this have anything to do with what @AstonMartin715 and I are experiencing?

Thanks!

e-ctrl

On 2/16/2023 8:10 AM, AstonMartin715 wrote:

@zzack01 https://github.com/zzack01 since a reboot of the Hoobs hub always solves the problem, at least for a while, that makes me suspicious that it would be an Insteon hub issue. Rebooting the Insteon hub does not solve it ever until the Hoobs hub is rebooted. Given everything I've seen, this feels like it's on the Hoobs/Plugin side of the equation. May be clutching at straws here but curious if there's a way to write a script so that the Hoobs hub sends a command to Insteon every 5 minutes to switch on or off a single source, and can then detect the status of that source back from Insteon, so we can identify more closely when the failure occurs and what might be causing it? Absent that I'm running out of ideas for diagnosis or process of elimination.

— Reply to this email directly, view it on GitHub https://github.com/kuestess/homebridge-platform-insteonlocal/issues/288#issuecomment-1433334481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOOUEGG663JV6F3SUEKTJDWXZGODANCNFSM6AAAAAAT264J2Y. You are receiving this because you were mentioned.Message ID: @.***>

kuestess commented 1 year ago

@greg-e-ctrl Nope - separate issue - see here: (#291)

AstonMartin715 commented 1 year ago

So I sent a light on command last night and it finally happened almost 24 hours later. It was preceded by a ‘Platform Insteonlocal BridgeWARNINGunhandled rejection: Error: read ECONNRESET’ message. See screenshot. FAD95B36-9556-4999-875A-2F78F49EAFB7

kuestess commented 1 year ago

@AstonMartin715 What is the cpu usage on the machine you're running Hoobs on?