search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.45k stars 30.69k forks source link

error at log: Timeout fetching xxx data and device is unavialable for few moments #73763

Closed KitHubek closed 1 year ago

KitHubek commented 2 years ago

The problem

See this timeout error at log, and devices are reallly on moment unavialable

Logger: homeassistant.components.xiaomi_miio Source: helpers/update_coordinator.py:197 Integration: Xiaomi Miio (documentation, issues) First occurred: 15 czerwca 2022, 08:54:49 (2358 occurrences) Last logged: 18:16:59

Timeout fetching fan1 data Timeout fetching purifer1 data Timeout fetching purifer2 data Timeout fetching fan2 data Timeout fetching purifer3 data Timeout fetching fan3 data

Purifers: 2x Mi 3H EU ver and Mi PRO EU ver Fans: 3x Smartmi Standing Fan 3

What version of Home Assistant Core has the issue?

2022.6.6

What was the last working version of Home Assistant Core?

2022.4.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

xiaomi_miio

Link to integration documentation on our website

https://www.home-assistant.io/integrations/xiaomi_miio/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching fan1 data
ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching fan2 data
ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching purifer1 data
ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching fan3 data
ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching purifer2 data
ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching purifer3 data

Additional information

No response

probot-home-assistant[bot] commented 2 years ago

Hey there @rytilahti, @syssi, @starkillerog, @bieniu, mind taking a look at this issue as it has been labeled with an integration (xiaomi_miio) you are listed as a code owner for? Thanks! (message by CodeOwnersMention)

xiaomi_miio documentation xiaomi_miio source (message by IssueLinks)

Maxi2509 commented 2 years ago

i have the same with my vacuums i am using.

Logger: homeassistant.components.xiaomi_miio Source: helpers/update_coordinator.py:197 Integration: Xiaomi Miio First occurred: 22. Juni 2022, 17:28:13 (5 occurrences) Last logged: 03:45:23

Timeout fetching roborock.vacuum.s6 data Timeout fetching Roborock S7 data Timeout fetching roborock.vacuum.a10 data

syssi commented 2 years ago

I'm looking for somebody who is able to setup a Home Assistant Core (python virtualenv) instance to compare the behavior of the pure HA core setup with a Home Assistant OS (docker) setup.

PskNorz commented 2 years ago

Same think all the time roborock s5 max goes unaivable all the time.

Logger: homeassistant.components.xiaomi_miio Source: helpers/update_coordinator.py:211 Integration: Xiaomi Miio First occurred: 9:46:39 PM (1 occurrences) Last logged: 9:46:39 PM

Timeout fetching Svetlana data

lbouriez commented 2 years ago

Hello, same since today with my S5 and S6, they keep becoming unavailable and I have a timeout in the logs:

Logger: homeassistant.components.xiaomi_miio
Source: helpers/update_coordinator.py:211
Integration: Xiaomi Miio (documentation, issues)
First occurred: 4:15:07 PM (8 occurrences)
Last logged: 4:51:47 PM

Timeout fetching Roborock S6 Pure data
Timeout fetching Living Room data
lbouriez commented 2 years ago

I need more testing but it seems my Unifi Dream machine was blocking some traffic from Xiaomi, that's why my 2 vacuums were having issues. My settings are these:

image

As soon as I stopped to block, everything is coming back online. For those having the issues, are you behind ubiquiti hardware ?

PskNorz commented 2 years ago

No i dont got ubiquitu hardware, my set up is simple, just a router but it keeps giving me errors.

Logger: homeassistant.components.xiaomi_miio Source: helpers/update_coordinator.py:211 Integration: Xiaomi Miio (documentation, issues) First occurred: July 19, 2022 at 12:06:41 AM (125 occurrences) Last logged: 9:10:37 AM

Timeout fetching xxxx data

dawiinci commented 2 years ago

I'm looking for somebody who is able to setup a Home Assistant Core (python virtualenv) instance to compare the behavior of the pure HA core setup with a Home Assistant OS (docker) setup.

I am using a pure python venv installation and have the same issue. In addition the temperature sensors often stay at the same level without any update for way too long.

My system is not reachable from the outside, but that does not apply since the devices are locally anyway (not using the cloud button).

Xiaomi Air purifiers H3 and Pro

dawiinci commented 2 years ago

The problem with not sending updates seems to be related to Xiaomi, since they seem to only give an update if the value changes at least 0.5 degree Celsius.

If that is on the plug-in side - which I hope - I would like to change that. Is that possible?

rytilahti commented 2 years ago

My system is not reachable from the outside, but that does not apply since the devices are locally anyway (not using the cloud button).

Not having them connected to the cloud might be irrelevant, as there are devices that are known to become unresponsive on specific conditions, like on some roborock gen1 firmware vacuums think they are stuck and are rebooted by a watchdog when they are not receiving any responses for their requests. See https://python-miio.readthedocs.io/en/latest/troubleshooting.html#intermittent-connection-issues-timeouts-xiaomi-vacuum for some details on that.

dawiinci commented 2 years ago

This was may concern. So there is nothing we can do, right? Maybe extending the timeout would be making sense, considering the watchdog has a timeout as well.

KitHubek commented 2 years ago

i agree with @dawiinci maby longer wait for response time will solve or partial solve problem? I have this problem with my three fans and three purifier. I checked and in XIaomi app this devices are all time avialable, so problem with unavialable devices exsist only in HA.

below last 24h, over 800 hits

Adnotacja 2022-08-04 231953

first three are fans fan.zhimi_fan_za5 last two pufifiers zhimi_airpurifier_mb3

Adnotacja 2022-08-04 231502

Enzokot commented 2 years ago

I'm looking for somebody who is able to setup a Home Assistant Core (python virtualenv) instance to compare the behavior of the pure HA core setup with a Home Assistant OS (docker) setup.

@syssi I can help with python virtualenv vs supervized and zhimi.humidifier.ca4, zhimi.humidifier.cb1, roborock.vacuum.s5. What I need to do?

syssi commented 2 years ago

@Enzokot Thanks for your support! Do you have the same issues mentioned above at your supervized setup?

Enzokot commented 2 years ago

@Enzokot Thanks for your support! Do you have the same issues mentioned above at your supervized setup?

Yes, with all listed devices

syssi commented 2 years ago

Please pick a debian based host and install the Home Assistant Core in a virtualenv: https://www.home-assistant.io/installation/generic-x86-64#install-dependencies

Please don't use any virtualization. For the test scenario it's important to have a super simple (bare metal) setup. If you have trouble to setup a virtualenv + homeassistant please let me know. I will try to provide support. As soon as the setup (HA core instance) is up and running (accessible via http://a.b.c.d:8123) we will your the Xiaomi devices. Let the setup run a few days, so we can compare the history graphs of both HA instances (virtualenv vs. dockerized=your production system).

Enzokot commented 2 years ago

@syssi I have new installation (without/simple any configs for HA) Home Assistant Core in a virtualenv on Raspbian PI4 and new one (without/simple any configs for HA) Home Assistant Supervized on mini PC like Gygabyte Brix (x86). This is good for testing? May be any tuning for the log file?

syssi commented 2 years ago

Perfect! So we are good to go. Please add the Xiaomi devices to your new setup. Make sure the setup is able to talk to your MiIO devices and please provide some screenshot (like above https://github.com/home-assistant/core/issues/73763#issuecomment-1205778133) of your history graphs. I assume there won't be so much dropouts compared to the supervized setup.

to4ko commented 2 years ago

same here - Purifier 2s became unavailable every few seconds Debian 11, HA Supervised 2022.8.2

image

fahmietalife commented 2 years ago

Same goes with my mi air purifier 3C, disconnect every few minutes

Logger: homeassistant.components.xiaomi_miio Source: helpers/update_coordinator.py:151 Integration: Xiaomi Miio (documentation, issues) First occurred: August 10, 2022 at 8:17:26 PM (55 occurrences) Last logged: 2:33:26 AM

Timeout fetching Mi Air Purifier 3C data

No issue at all with mi air purifier 3H

Enzokot commented 2 years ago

@syssi result by days 1 day: 1 2 day: 2 3 day: 3

syssi commented 2 years ago

Good job! It looks like there is no big difference between the setups. The supervized instance shows the same pattern as the virtualenv setup.

BillyFKidney commented 2 years ago

I need more testing but it seems my Unifi Dream machine was blocking some traffic from Xiaomi, that's why my 2 vacuums were having issues. My settings are these:

image

As soon as I stopped to block, everything is coming back online. For those having the issues, are you behind ubiquiti hardware ?

This integration is listed as LOCAL POLLING; why would location block affect the status?

BillyFKidney commented 2 years ago 
2022-08-12 11:39:33.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 11:42:11.004 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 11:47:02.004 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 11:56:10.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:02:50.001 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:06:11.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:11:04.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:11:37.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:14:56.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:17:35.007 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:21:06.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:21:25.006 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:27:27.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:33:16.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:42:39.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 12:59:49.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:01:50.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:04:51.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:06:04.003 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:12:00.001 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:26:38.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:39:42.001 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:45:32.002 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data
2022-08-12 13:46:50.005 ERROR (MainThread) [homeassistant.components.xiaomi_miio] Timeout fetching zhimi.airpurifier.mb3 data

I'm experiencing the same issue, but there's no pattern to the timing.

Interestingly, Ubiquiti states the device has been connected the entire time.

image
syssi commented 2 years ago

@BillyFKidney The firmware of some Xiaomi devices is pretty bad. The communication between HA/python-miio is local but the devices tries to talk to the Mi cloud periodically. If the internet access is blocked the firmware gets stuck or runs into timeouts. The local communication slows down or is impossible in these situations.

KitHubek commented 2 years ago

i have all communication allowed, and last 24h for my fan.zhimi_fan_za5 and two pufifiers zhimi_airpurifier_mb3

Adnotacja 2022-08-13 214014

syssi commented 2 years ago

@KitHubek There are devices out there which don't like to be polled very frequently. I don't know this fact does apply to your device. I just try to provide a much details as possible here.

KitHubek commented 2 years ago

last 7 days. Crazy is that Xiaomi purifer PRO has no problem. 7 days i cant see 1 error with purifer PRO. Other devices - no comment

Adnotacja 2022-08-22 001226

KitHubek commented 2 years ago

@syssi

what is the timeout set in the integration after which the device is considered unavailable? I did a regular fan ping test for 3 hours and the duration is max 3 seconds. Of course, the vast majority does not exceed 1s. I think that increasing the waiting time for a reply would solve the problem and it would definitely reduce it significantly

syssi commented 2 years ago

@KitHubek The default polling timeout is 10 seconds: https://github.com/home-assistant/core/blob/dev/homeassistant/components/xiaomi_miio/__init__.py#L78

IMO there is a difference between: A device responds to ICMP requests (ping) in time vs. the device API is able to respond to a miIO request. You could try to increase the timeout (by editing the file of you HA installation + restart). I assume this won't improve the situation because the miIO device will never respond to some requests.

KitHubek commented 2 years ago

since there are some inquiries to which the device will not answer, maybe it would be worth doing so that the device is simply not marked as unavailable based on these queries, or when available, take into account only those queries that are standard for everyone the type of device, i.e. the state of the on / off switch, ip address, anything else that is always present with each device

ainslied commented 2 years ago

Please pick a debian based host and install the Home Assistant Core in a virtualenv: https://www.home-assistant.io/installation/generic-x86-64#install-dependencies

Please don't use any virtualization. For the test scenario it's important to have a super simple (bare metal) setup. If you have trouble to setup a virtualenv + homeassistant please let me know. I will try to provide support. As soon as the setup (HA core instance) is up and running (accessible via http://a.b.c.d:8123) we will your the Xiaomi devices. Let the setup run a few days, so we can compare the history graphs of both HA instances (virtualenv vs. dockerized=your production system).

Hi @syssi , I have a core version has described in your message (on a beagle bone black which is a mini PC like the Raspberry PI) and unfortunately I have not the supervised version running in parrallel.

I have sometimes also some unavailabilities (see bellow), and that actually does not bother me. In this example the unavailability is only 22s : so who cares? image

But there is actually a critical side effect. I don't know if this occurs on the core version only or in all install versions but since these unavailabilities generate "ERRORS" it can break actually the run of other homeassistant processes or the core process itself. For example, if it happends during the startup of homeassistant, it will fail to load (see bellow). It can may be break also other services such as core updates or other critical processes ? image

So I see actually 2 issues :

Do you think I should create a new issue for this : catch the xiaomi_miio error (for example try to generate a warning and not an error?)

Regards, Ainslie

KitHubek commented 2 years ago

any update to fix this issue?

ntkoopman commented 1 year ago

I'm having this issue as well, and I can see the device reconnecting to the wifi in the hostapd logs after ~20 seconds. Since that means the entire connect is reset, it would explain why a timeout wouldn't work. After rebooting the vacuum it seems to be better for now (or maybe it will reconnect if it can't connect to the gateway?).

rytilahti commented 1 year ago

There is #82601 that might improve the situation.

@ntkoopman reconnection to wifi sounds like a device reboot which gives a (potentially useful?) clue that the device has arrived in an unrepairable state causing the reset. The root cause (if not fixed by the linked PR) remains still unknown :-(

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

rytilahti commented 1 year ago

Feel free to give the dev branch a try to see if the #82601 fixes (or at least alleviates) the issue, and please report back, thanks!

Daneish commented 1 year ago

Feel free to give the dev branch a try to see if the #82601 fixes (or at least alleviates) the issue, and please report back, thanks!

When testing, do I need to copy the entire xiaomi_miio folder from Dev into my local custom_components folder, or just the changed file (init.py)?