Flo by Moen integration goes offline many times per day

edjurkowski commented 10 months ago

The problem

I used an automation triggered by the Flo integration sensor Change system mode to report when Flo goes offline. It happens many times per day. I only hasve the water valve device in my integration.

What version of Home Assistant Core has the issue?

System Information version | core-2023.11.2 -- | -- installation_type | Home Assistant OS dev | false hassio | true docker | true user | root virtualenv | false python_version | 3.11.6 os_name | Linux os_version | 6.1.59 arch | x86_64 timezone | America/New_York config_dir | /config
Home Assistant Community Store
GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 5000 Installed Version | 1.33.0 Stage | running Available Repositories | 1333 Downloaded Repositories | 8

Home Assistant Cloud
logged_in | true -- | -- subscription_expiration | January 8, 2024 at 7:00 PM relayer_connected | true relayer_region | us-east-1 remote_enabled | true remote_connected | true alexa_enabled | false google_enabled | true remote_server | us-east-1-5.ui.nabu.casa certificate_status | ready can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok

Home Assistant Supervisor
host_os | Home Assistant OS 11.1 -- | -- update_channel | stable supervisor_version | supervisor-2023.11.3 agent_version | 1.6.0 docker_version | 24.0.6 disk_total | 30.8 GB disk_used | 11.8 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | Samba share (12.1.0), Advanced SSH & Web Terminal (16.0.1), File editor (5.7.0), Matter Server (4.10.2), Zigbee2MQTT (1.33.2-1), Mosquitto broker (6.3.1), Home Assistant Google Drive Backup (0.111.1), Studio Code Server (5.14.2), VLC (0.2.0), ESPHome (2023.11.1), Silicon Labs Flasher (0.2.0), Silicon Labs Multiprotocol (2.3.2)

Dashboards
dashboards | 3 -- | -- resources | 1 views | 15 mode | storage

Recorder
oldest_recorder_run | November 8, 2023 at 4:43 PM -- | -- current_recorder_run | November 16, 2023 at 6:08 PM estimated_db_size | 106.03 MiB database_engine | sqlite database_version | 3.41.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

https://www.home-assistant.io/integrations/flo/

Link to integration documentation on our website

No response

Diagnostics information

I am unable to drag file here.

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Water main Current system mode changed to home
8:08:15 AM - 3 hours ago
Water main Pending system alerts cleared (no problem detected)
8:08:15 AM - 3 hours ago
Water main Shutoff valve became unavailable
8:05:54 AM - 3 hours ago
Water main Current system mode became unavailable
8:05:54 AM - 3 hours ago
Water main Pending system alerts became unavailable
8:05:54 AM - 3 hours ago
Water main Shutoff valve turned on
7:07:29 AM - 4 hours ago
Water main Current system mode changed to home
7:07:29 AM - 4 hours ago
Water main Pending system alerts cleared (no problem detected)
7:07:29 AM - 4 hours ago
Water main Shutoff valve became unavailable
7:06:28 AM - 4 hours ago
Water main Current system mode became unavailable
7:06:28 AM - 4 hours ago
Water main Pending system alerts became unavailable
7:06:28 AM - 4 hours ago
Water main Shutoff valve turned on
6:07:23 AM - 5 hours ago
Water main Current system mode changed to home
6:07:23 AM - 5 hours ago
Water main Pending system alerts cleared (no problem detected)
6:07:23 AM - 5 hours ago
Water main Shutoff valve became unavailable
6:06:16 AM - 5 hours ago
Water main Current system mode became unavailable
6:06:16 AM - 5 hours ago
Water main Pending system alerts became unavailable
6:06:16 AM - 5 hours ago
Water main Shutoff valve turned on
5:20:35 AM - 6 hours ago
Water main Current system mode changed to home
5:20:35 AM - 6 hours ago
Water main Pending system alerts cleared (no problem detected)
5:20:35 AM - 6 hours ago
Water main Shutoff valve became unavailable
5:19:35 AM - 6 hours ago
Water main Current system mode became unavailable
5:19:35 AM - 6 hours ago
Water main Pending system alerts became unavailable
5:19:35 AM - 6 hours ago
Water main Shutoff valve turned on
5:07:24 AM - 6 hours ago
Water main Current system mode changed to home
5:07:24 AM - 6 hours ago
Water main Pending system alerts cleared (no problem detected)
5:07:24 AM - 6 hours ago
Water main Shutoff valve became unavailable
5:06:18 AM - 6 hours ago
Water main Current system mode became unavailable
5:06:18 AM - 6 hours ago
Water main Pending system alerts became unavailable
5:06:18 AM - 6 hours ago
Water main Shutoff valve turned on
4:08:27 AM - 7 hours ago
Water main Current system mode changed to home
4:08:27 AM - 7 hours ago
Water main Pending system alerts cleared (no problem detected)
4:08:27 AM - 7 hours ago
Water main Shutoff valve became unavailable
4:06:06 AM - 7 hours ago

Additional information

No response

home-assistant[bot] commented 10 months ago

Hey there @dmulcahey, mind taking a look at this issue as it has been labeled with an integration (flo) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `flo` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign flo` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

flo documentation flo source _{^{(message by IssueLinks)}}

edjurkowski commented 10 months ago

Referenced entities switch.water_main_shutoff_valve are missing or not currently available 11:07:23 AM – (WARNING) helpers/service.py - message first occurred at 11:07:20 AM and shows up 2 times Unexpected error fetching flo-235639b9-5f21-4fcb-88d4-2504e608c553 data: unexpected character: line 1 column 1 (char 0) 11:07:13 AM – (ERROR) Flo - message first occurred at November 16, 2023 at 7:07:26 PM and shows up 16 times Timeout fetching flo-235639b9-5f21-4fcb-88d4-2504e608c553 data 10:05:53 AM – (ERROR) Flo - message first occurred at November 16, 2023 at 7:06:14 PM and shows up 11 times

edjurkowski commented 10 months ago

I enabled the Flo integration debug logging. The output file is 239 MB (251,109,409 bytes). I ran it for about 5 minutes. Notepad had a problems handling it. Wyze dialog in the log file.

2023-11-18 14:17:19.022 DEBUG (MainThread) [wyzeapy.wyze_auth_lib] Request: 2023-11-18 14:17:19.022 DEBUG (MainThread) [wyzeapy.wyze_auth_lib] url: https://api.wyzecam.com/app/v2/device/get_property_list 2023-11-18 14:17:19.022 DEBUG (MainThread) [wyzeapy.wyze_auth_lib] json: {'phone_system_type': '1', 'app_version': '2.18.43', 'app_ver': 'com.hualai.WyzeCam___2.18.43', 'sc': '9f275790cab94a72bd206c8876429f3c', 'ts': 1700335038, 'sv': '9d74946e652647e9b6c9d59326aef104', 'access_token': 'Sanitized', 'phone_id': 'b2f1203b-99fe-40e1-bbd9-838edeb13e7d', 'app_name': 'com.hualai.WyzeCam', 'device_model': 'WYZE_CAKP2JFUS', 'device_mac': '7C78B22480E9', 'target_pid_list': []} 2023-11-18 14:17:19.022 DEBUG (MainThread) [wyzeapy.wyze_auth_lib] headers: None

edjurkowski commented 10 months ago

home-assistant_flo_2023-11-19T18-23-30.008Z.log

I removed the wyze intgration and pulled a new log. I deleted the Wyze folder from the /homeassistant/custom_components folder. The wyzeapi dialog is gone. I still see gaps in Water main Current system mode.

edjurkowski commented 10 months ago

dhi20 commented 9 months ago

I appear to have exactly the same issue. Disconnects are almost exactly every 55 min.

celliott13 commented 9 months ago

I appear to have exactly the same issue. Disconnects are almost exactly every 55 min.

Same with my setup as well, 55 minutes like clockwork it disconnects

edjurkowski commented 9 months ago

Welcome to the Flo problem.

edjurkowski commented 9 months ago

Hello, Is there any activity going on for this issue? Thank you.

woofcub1 commented 8 months ago

I seem to have the same issue. Disconnects every 57 minutes or so.

basilisk487 commented 7 months ago

The problem is on Flo's API side, the endpoint (https://api-gw.meetflo.com) is timing out. Looks like their load balancer timeout is very low (5 seconds), so if the request takes longer than that, load balancer returns a HTTP 502. My guess would be that this happens when they are running hourly data aggregation on schedule. There's not much we can do about the API itself, but what would really help is if the integration could be configured to ignore a single error (ideally, up to N consecutive errors) before marking itself as unavailable.

This is the actual response body from the API:

<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.25.3</center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->

dmulcahey commented 7 months ago

The problem is on Flo's API side, the endpoint (https://api-gw.meetflo.com) is timing out. Looks like their load balancer timeout is very low (5 seconds), so if the request takes longer than that, load balancer returns a HTTP 502. My guess would be that this happens when they are running hourly data aggregation on schedule. There's not much we can do about the API itself, but what would really help is if the integration could be configured to ignore a single error (ideally, up to N consecutive errors) before marking itself as unavailable.

This is the actual response body from the API:
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.25.3</center>
</body>
</html>

done

home-assistant / core