search

kevinvincent / ha-wyzesense

A Home Assistant Component to interface with the WYZE Sense hub and sensor system
369 stars 101 forks source link

[Errno 110] Operation timed out #20

Open Nighteyez07 opened 5 years ago

Nighteyez07 commented 5 years ago

Environment: HASSIO 0.97.1 via a VirtualBox on Windows 7. Receiver is passed through and accessible at /dev/hidraw1.

Plugin installed via HACS running 0.0.5 of Wyze Sense Component

Issue: Wyzesense works perfectly and displays messages every 15 minutes or so until it I get a message that says [Errno 110] Operation timed out and from there I get absolutely no message from Wyzesense unless I restart HA. Included is a PasteBin that shows a log with responses until the error. Nothing included after the error due to size restriction from PasteBin

https://pastebin.com/FHcjJ3Zc

I do have the logging set to debug for the wyzesense gateway and custom_components.wyzesense.

kevinvincent commented 5 years ago

Okay yeah that happens when theres a low level error communicating with the hub. It looks like im going to have to add in more robust retrying. Thanks for the report. It currently will exit when it runs into any one of these low level errors so a restart is necessary.

Nighteyez07 commented 5 years ago

If there's anything you'd like me to try on my end to test an update I'm more than happy to. Otherwise I shall eagerly await for the updated payload on HACS.

kandbcontrols commented 5 years ago

@kevinvincent Hi Kevin. I was just getting ready to open an issue with this same problem. I also have the log file. I am running VirtualBox on Windows 10 Home Assistant 0.98.1 and the plugin installed via HACS 0.0.5. Mine will run for about 3-4 hours before getting the [Errno 110]. Do you want me to still open up an issue? BTW - Thank You for this wonderful plugin and your hard work. I know it will be great!

aaronpk commented 5 years ago

I was running this successfully for about a week, then I started getting timeout errors a few hours after starting up home assistant. I was a couple minor releases behind, so I updated to the current version a couple days ago and it's gotten way worse now. It won't stay connected for more than a half hour, and half the time it fails to initialize the devices when starting up HA as well.

kandbcontrols commented 5 years ago

I just let mine go because I am out of town and figured I would wait for Kevin to update for this issue. However I check my home every day and the Wyze node started talking again without me having to restart HA. That was 6 days ago and it has continued to keep talking. IDK. I just hope it keeps going because I love these sensors.

Nighteyez07 commented 5 years ago

I made a manual edit to my wyzesense/wyzesense_custom.py file. I changed the dongle timeout back to 2 minutes from the 5 minutes that it was set to in the latest version. I've been running that change for the last week and haven't had any issues with timeouts. I might try upping it to 3 just to figure out the upper bounds.

class Dongle(object):
    _CMD_TIMEOUT = 2
kandbcontrols commented 5 years ago

@Nighteyez07 - Any luck with the timeout change? I am thinking of making the same change. What did you end up with? Has it continued to work?

Nighteyez07 commented 5 years ago

@Nighteyez07 - Any luck with the timeout change? I am thinking of making the same change. What did you end up with? Has it continued to work?

I changed it to 2 minutes and only once so far has the problem happened again that I know of. If I could find a way to get an alert on a specific message popping up in the log then I would know more for sure.

mattyb8562 commented 5 years ago

I was on my way to report this same issue, I see I'm not the only one. KevinVincent, it looks like you're already aware of the problem. Let me know if you still need some logs and I'll be glad to submit them.

As there seems to be some success by editing the .py I have made the changed mentioned above. Hopefully, that'll get me by for a while.

mattyb8562 commented 4 years ago

Well, editing the timeout didn't help. I've had Errno110 error a couple of times since.

I have created an automation that will hopefully restart HA when the error is detected as a temporary workaround.

aaronpk commented 4 years ago

I would love if there's a way to restart just this component instead of the entire HA stack, as that's kind of disruptive. Anyone know if there's a way to do that.

For the others who are having this problem, is the dongle attached to a USB hub or directly to a USB port on the computer? Mine is connected to a hub because it's too physically large to connect directly to a port, but I'm curious if that might help make the problems go away.

mattyb8562 commented 4 years ago

Mine is connected directly to the USB port. I don't know of a way to restart just the component. The homeassistant.restart service was the easiest thing I could think of. A random restart once a day is less disruptive than having the sensors stop randomly in my opinion.

aaronpk commented 4 years ago

If I can get them stable for an entire day I may do that too. Right now I'm just not using them because they disconnect all the time which is really sad because I had some nice automation rules set up that rely on these sensors.

Nighteyez07 commented 4 years ago

My dongle is connected via a USB extender cable that allows me to mount the receiver to the ceiling. And the cable is directly connected to a USB port on the back of the server. @mattyb8562 are you just scheduling a once a day restart or do you have logging detection that then does the restart?

Nighteyez07 commented 4 years ago

This weekend I may take a look at something like this to check the log for the error and then do a restart. It appears there's no method currently in HA to restart just a component without restarting the entire HA instance. But that the HA developers are looking to add that capability. https://community.home-assistant.io/t/restart-single-platform-or-component-possible/23721/2

mattyb8562 commented 4 years ago

In theory when the error occurs HA should restart. I'm still waiting for the error again to verify it works.

alias: wyze error fix trigger:

mattyb8562 commented 4 years ago

FYI my automation didn't work. It looks like it's going to be more difficult than expected.

MightyGood commented 4 years ago

FYI my automation didn't work. It looks like it's going to be more difficult than expected.

Looked into this and found: https://www.home-assistant.io/integrations/system_log

Above the examples it says, "Here are some examples using the events posted by system_log. fire_event must be set to true for these to work."

So I added - 

system_log:
  fire_event: true

to my configuration.yaml. Then it started firing events from the system log. I haven't had the error come up yet, but I'll update if it restarts when it does happen.

mattyb8562 commented 4 years ago

@Musicman247 Thanks! I'll check it out. I missed the config part when I was looking into it, I'm glad you caught it.

MightyGood commented 4 years ago

We'll, I just got a notification that HA restarted because of the error. Here is the system log event in case you want to specifically call out the source to narrow down when to call for a restart (Restart is the title of the PushBullet notification I set up):

Restart: {"event_type":"system_log_event","event":{"first_occured":1571365461.1773076,"timestamp":1571365461.1773076,"level":"ERROR","message":"[Errno 110] Operation timed out","exception":"","root_cause":null,"source":"custom_components/wyzesense/wyzesense_custom.py","count":1}}

So this means I can rest easy knowing that my Wyzesense sensors should only ever be down a handful of seconds while HA restarts!

kandbcontrols commented 4 years ago

Are you using

logger: default: critical logs: custom_components.wyzesense: debug wyzesense.gateway: debug

I just added this automation.

script:

restartha: alias: "Restart HA if Wyzesense error detected" sequence:

It seemed to me when i added the custom_components.wyzesense: debug before, I got the error way more often then when it wasn't enabled. If i could limit a restart to once every few days or so I could live with that.

On Thu, Oct 17, 2019 at 11:19 PM Musicman247 notifications@github.com wrote:

We'll, I just got a notification that HA restarted because of the error. Here is the system log event in case you want to specifically call out the source to narrow down when to call for a restart (Restart is the title of the PushBullet notification I set up:

Restart: {"event_type":"system_log_event","event":{"first_occured":1571365461.1773076,"timestamp":1571365461.1773076,"level":"ERROR","message":"[Errno 110] Operation timed out","exception":"","root_cause":null,"source":"custom_components/wyzesense/wyzesense_custom.py","count":1}}

So this means I can rest easy knowing that my Wyzesense sensors should only ever be down a handful of seconds while HA restarts!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kevinvincent/ha-wyzesense/issues/20?email_source=notifications&email_token=ALMSZZJ3BZEPLWPVM3B74KTQPETMVA5CNFSM4IP55HJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBSLTJY#issuecomment-543472039, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMSZZPD3G45IUY3XNCLJ5TQPETMVANCNFSM4IP55HJQ .

kandbcontrols commented 4 years ago

Would you mind posting you automation? I just got the Erno 110 and it did not restart HA.

Thx

On Mon, Oct 21, 2019 at 8:01 AM Kevin Burns kandbcontrols@gmail.com wrote:

Are you using

logger: default: critical logs: custom_components.wyzesense: debug wyzesense.gateway: debug

I just added this automation.

  • alias: Restart HA trigger:
    • event_data: {} event_type: system_log_event platform: event condition:
    • condition: template value_template: '''{{ "[Errno 110] Operation timed out" in trigger.event.data.message }}'''
    • condition: state entity_id: script.restartha state: 'off' action: service: homeassistant.turn_on entity_id: script.restartha

script:

restartha: alias: "Restart HA if Wyzesense error detected" sequence:

  • delay: minutes: 1
  • service: notify.pushbullet data: message: 'Wyzesense Error, restarting HA'
  • service: homeassistant.restart

It seemed to me when i added the custom_components.wyzesense: debug before, I got the error way more often then when it wasn't enabled. If i could limit a restart to once every few days or so I could live with that.

On Thu, Oct 17, 2019 at 11:19 PM Musicman247 notifications@github.com wrote:

We'll, I just got a notification that HA restarted because of the error. Here is the system log event in case you want to specifically call out the source to narrow down when to call for a restart (Restart is the title of the PushBullet notification I set up:

Restart: {"event_type":"system_log_event","event":{"first_occured":1571365461.1773076,"timestamp":1571365461.1773076,"level":"ERROR","message":"[Errno 110] Operation timed out","exception":"","root_cause":null,"source":"custom_components/wyzesense/wyzesense_custom.py","count":1}}

So this means I can rest easy knowing that my Wyzesense sensors should only ever be down a handful of seconds while HA restarts!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kevinvincent/ha-wyzesense/issues/20?email_source=notifications&email_token=ALMSZZJ3BZEPLWPVM3B74KTQPETMVA5CNFSM4IP55HJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBSLTJY#issuecomment-543472039, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMSZZPD3G45IUY3XNCLJ5TQPETMVANCNFSM4IP55HJQ .

mattyb8562 commented 4 years ago

@kandbcontrols I wish I could help you. It's not firing for me either. Can't get Node-Red to play nice either.

kandbcontrols commented 4 years ago

I ended up getting it to work with the above code minus two quote's. However, like I mentioned, when I have the logger enabled with wyze debug, it seems to cause it to timeout way more often. Yesterday HA re-started 4 times due to the timeout. I really liked these sensors and went all in with them, but they are just not trust worthy. My wife was really starting to get annoyed with me. I removed them and went with Aqara door and motion sensors. I have been testing them for a few weeks now with 100% uptime. I'm not sure what to do with all the Wyze motion and door sensors I have now. Maybe a fix will be found, but for now they are going in a drawer.

BTW here is the automation that works.

config.yaml

logger: default: critical logs: custom_components.wyzesense: debug wyzesense.gateway: debug

automation.yaml

script.yaml

restartha: alias: "Restart HA if Wyzesense error detected" sequence:

On Fri, Oct 25, 2019 at 8:23 AM mattyb8562 notifications@github.com wrote:

@kandbcontrols https://github.com/kandbcontrols I wish I could help you. It's not firing for me either. Can't get Node-Red to place nice either.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kevinvincent/ha-wyzesense/issues/20?email_source=notifications&email_token=ALMSZZKAK2HRYZE6DM7M6W3QQLQLNA5CNFSM4IP55HJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECIF5GY#issuecomment-546332315, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMSZZPRFKO4W6E2MRVAJFDQQLQLNANCNFSM4IP55HJQ .

mattyb8562 commented 4 years ago

@kandbcontrols Have you looked gosense for wyze sense? https://github.com/dariopb/ha-gosenseapp I think I'm going to give it a try and see if I can get better results. If you end up not using your sensers I may be interested in taking them off your hands :)

mattyb8562 commented 4 years ago

This is working to restart HA when the Errno 110 occurs. No need to enable wyze debug. On another note, I have only had the error once in the past 5 days instead of 1 or 2 times per day. A while ago I had a contact sensor the had fallen and broke the read switch. I took the battery out and left it on my desk with intentions of repurposing it as a leak sensor. The timing seems to be about right when the Errno 110 error started happening. Anyway, I removed the sensor from the HA wyze component and the error stopped until last night. This has me thinking that part of the issue is the component looking for that sensor. I would assume sensors with low battery or poor signal could cause the same problem.

configuration.yaml

'system_log: fire_event: true`

automations.yaml

` alias: Wyze restart trigger:

aaronpk commented 4 years ago

Just wanted to chime in again to say that I moved my entire Home Assistant install off the Raspberry Pi and onto an Intel NUC, and I have now been running this project successfully for over a week with no timeouts or other errors!

liquidzyklon commented 4 years ago

My other details are mentioned in post #78

Configuration: NUC running Virtualbox Hass.IO on HassOS 2.11 Home Assistant Version 0.103.4 Installed HA-Wyzesense 0.0.7 from HACS I do have six Wyze Sense sensors: 2 Motion, 4 contacts

https://pastebin.com/HNmSUFJM

With the update the stability of this custom components has improve greatly. I still managed to get this error message again. In the paste, I can get this error message after a few times of testing the sensors (~10 times). With the latest 0.0.7, I am able to test it for a few minutes (>50 times) before I got the error message.

liquidzyklon commented 4 years ago

Just a random thought. Do we know which firmware that the Wyze Dongles are?

If the author (Kevin) has one of the earlier versions and he does not experience this type of error message. I am assuming he hasn't plugged the dongle back into a Wyze camera which can possibly update the firmware.

Whereas most users bought the WyzeSense after Kevin and likely plugged it into the WyzeCam to make sure the sensors all work. In doing so, the firmware on the dongle could have been upgraded.

photinus commented 4 years ago

@liquidzyklon I have updated the dongle a couple times, Not sure where I landed/stopped updating, but have been through a couple.

kevinvincent commented 4 years ago

Hi everyone, I'll try to do a deep dive on this tonight. I have unfortunately been unable to reproduce it (hmm maybe I should try unplugging the dongle while HA is running) but will try more.

If anyone is able to reliably reproduce this here are some debugging things that will help me:

  1. Please note if the LEDs on the dongle change after this error occurs
  2. After the error occurs please run the unix command dmesg and post the output. This will let me see if theres any usb disconnect / reconnect or usb driver issues going on as well.

EDIT: I moved my dongle to a usb hub (instead of a direct connection to my machine) and have started seeing similar errors. Should make it easier to debug now that I can somewhat reproduce it.

singhj0910 commented 4 years ago

I just recently started getting this error too. ([Errno 110] Operation timed out). A restart only allows it to work for a few minutes before the error pops up again. It's been working well for me since I set it up maybe 3 months ago up until maybe a week ago when the error started happening. I haven't changed anything that I can remember. The hub has been hooked up to a USB extension cable the whole time. HA is running in a Ubuntu VM on Virtualbox through docker.

  1. The light on the hub remains blue after the error occurs. It only ever changes to amber if I unplug it and plug it back in.

  2. dmesg output: https://pastebin.com/uGGtaabp

I'm glad you are aware of it. Hopefully you can figure it out because I, too, like these sensors.

finnzz commented 4 years ago

I was getting the [Errno 110] Operation timed out every day or two.

I'm using Virtualbox 6 on Windows 10. I think that there may be a problem with the USB2 eHCI controller in Virtualbox. I have seen posts in the VB forums saying the eHCI controller is the least stable. I switched to the xHCI USB3 controller, and haven't seen any '[Errno 110] Operation timed out' issues for 5 days now.

I've noticed there are a lot of people experiencing this problem using Virtualbox, and it may be worth trying. You don't even need USB3.0 ports on your host computer to use the xHCI controller in Virtualbox.

singhj0910 commented 4 years ago

I was getting the [Errno 110] Operation timed out every day or two.

I'm using Virtualbox 6 on Windows 10. I think that there may be a problem with the USB2 eHCI controller in Virtualbox. I have seen posts in the VB forums saying the eHCI controller is the least stable. I switched to the xHCI USB3 controller, and haven't seen any '[Errno 110] Operation timed out' issues for 5 days now.

I've noticed there are a lot of people experiencing this problem using Virtualbox, and it may be worth trying. You don't even need USB3.0 ports on your host computer to use the xHCI controller in Virtualbox.

I can confirm this is indeed the solution for VB users. I saw this solution in the community forums the other day. If you posted that as well then thank you. I would have never figured that one out.

I was actually going to post this here later today as I have been testing it to see if it would drop again. It's been 3 days now and not one Errno 110. All sensors are responding as they should.

finnzz commented 4 years ago

Ah no, I never found that to thread, but I'm glad there is a solution. It would be a good idea to add this tidbit to the troubleshooting documentation here

kandbcontrols commented 4 years ago

When you switch to the xHC1 what do you change your configuration.yaml to? From "/dev/hidraw1" to ????

Thanks

On Fri, Mar 6, 2020 at 6:08 PM finnzz notifications@github.com wrote:

Ah no, I never found that to thread, but I'm glad there is a solution. It would be a good idea to add this tidbit to the troubleshooting documentation here

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kevinvincent/ha-wyzesense/issues/20?email_source=notifications&email_token=ALMSZZJQEL7QCGNBLN5JWYLRGF653A5CNFSM4IP55HJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEODEOJQ#issuecomment-596002598, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMSZZLDYAM35M2AFMBYSFTRGF653ANCNFSM4IP55HJQ .

finnzz commented 4 years ago

Switching between eHCI and xHCI Virtualbox controllers shouldn't require you to change anything in your config yaml.

But I have mine set to 'auto' so I don't have to worry about the device changing names between hidraw0, hidraw1, hidraw2.

configuration.yaml:

finnzz commented 4 years ago

By the way, since many of you may be running Windows hosts with your virtualbox. You may be interested to know that you can use devcon.exe to disable and enable the Wyze bridge so that you don't have to unplug/plug it after every windows reboot.

I rambled on about it here: https://github.com/kevinvincent/ha-wyzesense/issues/8#issuecomment-595578003

kandbcontrols commented 4 years ago

finzz,

I started using this integration when it first came and I bought 8 door sensors and 5 motion sensors. I love them! However, they would stop working daily with the [Errno 110] error. I tried many suggestions from the forums with no success. I finally gave up and removed all of it from HA. When I read your post, I got excited. I have them back in and have my fingers crossed. I just wanted to say thank you!!! It's people like you and kevinvincent that make this possible for the rest of us. I will update after a couple of days.

Thank You!!!!!!

On Fri, Mar 6, 2020 at 8:54 PM finnzz notifications@github.com wrote:

By the way, since many of you may be running Windows hosts with your virtualbox. You may be interested to know that you can use devcon.exe to disable and enable the Wyze bridge so that you don't have to unplug/plug it after every windows reboot.

I rambled on about it here:

8 (comment)

https://github.com/kevinvincent/ha-wyzesense/issues/8#issuecomment-595578003

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kevinvincent/ha-wyzesense/issues/20?email_source=notifications&email_token=ALMSZZIQA7GRQCNWQTVQDWLRGGSPFA5CNFSM4IP55HJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEODLRLQ#issuecomment-596031662, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMSZZP75W46SW5DGCH2EGDRGGSPFANCNFSM4IP55HJQ .

finnzz commented 4 years ago

Thanks, happy to help :). Local control of my Wyze sensors was the reason I installed Home Assistant so my hat off to Kevin as well for his work.