search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.69k stars 29.96k forks source link

Shelly Integration, all Generation 1 devices unavailable and cannot get them to re-connect #75118

Closed xrsprint closed 2 years ago

xrsprint commented 2 years ago

The problem

Hi there

This morning i had to reboot my PC for updated which is running hypervisor and HA on a VM, once restarted i noticed all of my Generation 1 shelly devices unavailable (i have about 30 of them), all Generation 2 devices were still online, after rebooting a number of times, i then rebooted all shelly devices with no luck, then rebooted everything from Switches, Wifi, Gateway, VM Host, and still no luck

i first verified all of my gen 1 devices are running the current firmware of 20220209-094317/v1.11.8-g8c7bb8d and all also have the Colo it enabled pointing back to my HA server which is 192.168.X.X:5683

i then deleted onc shelly device and tried to re-add and got the following message

2

note that if i delete a shelly device and then reboot, HA does auto discover it

i then factory reset the shelly unit, and still did the same issue as above . i then tried adding to configration.yaml a line to change the ColoIT port and then set it to the new port on a shelly, and no luck there either

i then decided to install a new install of HA on my VM (using the same hyper-v instance but with a new virtual disk,) meaning it kept the same IP, and then restored from a backup which was done two days ago when these were working with no luck, i then done another fresh install and starting a new HA system, which also done the same, and then reverted back to a older HA image i had with 8.1 and this also failed. i also tried changing IP addresses of the HA but keeping the same MAC of the VM and also no help there

after the tests above i believe i have ruled out the shellys and HA but the issue is still there, i can log into the units via web and control them with no issues and also i can confirm there is no firewall or blocking on the network as all these devices are on the same subnet

any help here would be greatly appreciated

Thanks

What version of Home Assistant Core has the issue?

core-2022.7.3

What was the last working version of Home Assistant Core?

core-2022.7.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Shelly

Link to integration documentation on our website

https://www.home-assistant.io/integrations/shelly

Diagnostics information

logs turned off

Example YAML snippet

N/A as its an integrated setup

Anything in the logs that might be useful for us?

2022-07-13 20:26:40 WARNING (MainThread) [homeassistant.config] Key whitelist_external_dirs has been replaced with allowlist_external_dirs. Please update your config
2022-07-13 20:26:41 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=630 from 2022-07-13 10:23:05)
2022-07-13 20:26:49 ERROR (MainThread) [snitun.client.client_peer] Challenge/Response error with SniTun server (0 bytes read on a total of 32 expected bytes)
2022-07-13 20:26:49 ERROR (MainThread) [hass_nabucasa.remote] Connection problem to snitun server
2022-07-13 20:26:52 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-PL1PM-DISHWASHER' for shelly integration not ready yet: Cannot connect to host 192.168.199.174:80 ssl:default [Connect call failed ('192.168.199.174', 80)]; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-EM' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-SOLAR2' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-SOLAR1' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-KITCHEN' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-DIM2-LOUNGE' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-LAUNDRY' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-MASTER' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-BEDSIDE' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-HALLWAY' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-BEDROOM2' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-SPRINT' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-BEDROOM1' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-ROBE' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-FRONTDOOR' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-BATHROOM' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-BATHROOM' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-ENSUITE2' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-ENSUITE1' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-WASHING' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-ROLLER TRIGGER' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-BATHROOM-ALEXA' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-1PM-GARAGE-ALEXA' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-TOILET' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-2.5-GARAGE-TV' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-AUTOMATION-4' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-AUTOMATION-3' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-AUTOMATION-2' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-AUTOMATION-1' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-COMMS FAN' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:26:58 WARNING (MainThread) [homeassistant.config_entries] Config entry 'SHELLY-UNI-LIZARD' for shelly integration not ready yet: Timeout during device setup; Retrying in background
2022-07-13 20:27:04 ERROR (MainThread) [snitun.client.client_peer] Challenge/Response error with SniTun server (0 bytes read on a total of 32 expected bytes)
2022-07-13 20:27:04 ERROR (MainThread) [hass_nabucasa.remote] Connection problem to snitun server
2022-07-13 20:27:06 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.210 initialize error - TimeoutError()
2022-07-13 20:27:20 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.157 initialize error - TimeoutError()
2022-07-13 20:27:20 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.167 initialize error - TimeoutError()
2022-07-13 20:27:20 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.231 initialize error - TimeoutError()
2022-07-13 20:27:21 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.164 initialize error - TimeoutError()
2022-07-13 20:27:21 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.210 initialize error - TimeoutError()
2022-07-13 20:27:21 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.170 initialize error - TimeoutError()
2022-07-13 20:27:21 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.175 initialize error - TimeoutError()
2022-07-13 20:27:22 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.234 initialize error - TimeoutError()
2022-07-13 20:27:23 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.203 initialize error - TimeoutError()
2022-07-13 20:27:25 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.204 initialize error - TimeoutError()
2022-07-13 20:27:25 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.160 initialize error - TimeoutError()
2022-07-13 20:27:25 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.171 initialize error - TimeoutError()
2022-07-13 20:27:25 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.233 initialize error - TimeoutError()
2022-07-13 20:27:25 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.172 initialize error - TimeoutError()
2022-07-13 20:27:26 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.165 initialize error - TimeoutError()
2022-07-13 20:27:27 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.228 initialize error - TimeoutError()
2022-07-13 20:27:28 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.226 initialize error - TimeoutError()
2022-07-13 20:27:28 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.168 initialize error - TimeoutError()
2022-07-13 20:27:36 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.210 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.226 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.217 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.234 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.233 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.232 initialize error - TimeoutError()
2022-07-13 20:27:39 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.207 initialize error - TimeoutError()
2022-07-13 20:27:40 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.204 initialize error - TimeoutError()
2022-07-13 20:27:40 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.160 initialize error - TimeoutError()
2022-07-13 20:27:40 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.171 initialize error - TimeoutError()
2022-07-13 20:27:40 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.172 initialize error - TimeoutError()
2022-07-13 20:27:41 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.165 initialize error - TimeoutError()
2022-07-13 20:27:42 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.228 initialize error - TimeoutError()
2022-07-13 20:27:43 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.168 initialize error - TimeoutError()
2022-07-13 20:27:44 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.203 initialize error - TimeoutError()
2022-07-13 20:27:44 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.173 initialize error - TimeoutError()
2022-07-13 20:27:44 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.211 initialize error - TimeoutError()
2022-07-13 20:27:44 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.197 initialize error - TimeoutError()
2022-07-13 20:27:46 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.206 initialize error - TimeoutError()
2022-07-13 20:27:46 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.166 initialize error - TimeoutError()
2022-07-13 20:27:47 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.159 initialize error - TimeoutError()
2022-07-13 20:27:47 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.213 initialize error - TimeoutError()
2022-07-13 20:27:47 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.169 initialize error - TimeoutError()
2022-07-13 20:27:48 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.161 initialize error - TimeoutError()
2022-07-13 20:27:48 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.231 initialize error - TimeoutError()
2022-07-13 20:27:49 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.232 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.157 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.167 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.204 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.226 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.217 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.234 initialize error - TimeoutError()
2022-07-13 20:27:50 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.233 initialize error - TimeoutError()
2022-07-13 20:27:51 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.164 initialize error - TimeoutError()
2022-07-13 20:27:51 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.210 initialize error - TimeoutError()
2022-07-13 20:27:51 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.170 initialize error - TimeoutError()
2022-07-13 20:27:51 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.175 initialize error - TimeoutError()
2022-07-13 20:27:52 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.207 initialize error - TimeoutError()
2022-07-13 20:27:55 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.160 initialize error - TimeoutError()
2022-07-13 20:27:55 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.171 initialize error - TimeoutError()
2022-07-13 20:27:55 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.172 initialize error - TimeoutError()
2022-07-13 20:27:56 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.165 initialize error - TimeoutError()
2022-07-13 20:27:57 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.228 initialize error - TimeoutError()
2022-07-13 20:27:58 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.168 initialize error - TimeoutError()
2022-07-13 20:27:59 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.203 initialize error - TimeoutError()
2022-07-13 20:28:06 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.210 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.217 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.234 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.233 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.231 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.232 initialize error - TimeoutError()
2022-07-13 20:28:09 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.207 initialize error - TimeoutError()
2022-07-13 20:28:10 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.160 initialize error - TimeoutError()
2022-07-13 20:28:10 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.171 initialize error - TimeoutError()
2022-07-13 20:28:10 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.226 initialize error - TimeoutError()
2022-07-13 20:28:10 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.172 initialize error - TimeoutError()
2022-07-13 20:28:11 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.165 initialize error - TimeoutError()
2022-07-13 20:28:12 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.204 initialize error - TimeoutError()
2022-07-13 20:28:12 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.228 initialize error - TimeoutError()
2022-07-13 20:28:13 WARNING (MainThread) [aioshelly.block_device] device 192.168.199.168 initialize error - TimeoutError()

Additional information

System Health

version core-2022.7.3
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.10.5
os_name Linux
os_version 5.15.45
arch x86_64
timezone Australia/Sydney
config_dir /config
Home Assistant Cloud logged_in | true -- | -- subscription_expiration | 15 July 2022 at 10:00 relayer_connected | true remote_enabled | true remote_connected | true alexa_enabled | false google_enabled | false remote_server | ap-southeast-1-0.ui.nabu.casa can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Home Assistant Supervisor host_os | Home Assistant OS 8.2 -- | -- update_channel | stable supervisor_version | supervisor-2022.07.0 agent_version | 1.2.1 docker_version | 20.10.14 disk_total | 30.8 GB disk_used | 3.8 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | Samba share (10.0.0), File editor (5.3.3), MariaDB (2.4.0), Check Home Assistant configuration (3.11.0)
Dashboards dashboards | 1 -- | -- resources | 0 views | 5 mode | storage
Recorder oldest_recorder_run | 7 July 2022 at 09:10 -- | -- current_recorder_run | 13 July 2022 at 20:26 estimated_db_size | 207.78 MiB database_engine | mysql database_version | 10.4.19
probot-home-assistant[bot] commented 2 years ago

shelly documentation shelly source (message by IssueLinks)

probot-home-assistant[bot] commented 2 years ago

Hey there @balloob, @bieniu, @thecode, @chemelli74, mind taking a look at this issue as it has been labeled with an integration (shelly) you are listed as a code owner for? Thanks! (message by CodeOwnersMention)

thecode commented 2 years ago

I had very bad experience with running windows Hypervisor and I don't recommended this for running HA under VM. As you did not change anything in your setup or HA I believe the CoAP packets are not reaching HA.

For debugging it please enable debug logging, restart HA and provide the home-assistant.log and attached the log few minutes after restart: To do so add the following to your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.shelly: debug
    aioshelly: debug

Further if possible please download and provide the diagnostics data for this integration.

Note: it is better to drag the log into the comment (which will add it as an attachment) and not copy paste as it is hard to read logs in GitHub.

Steph-74 commented 2 years ago

Just as an additional info: I updated 2 hours ago to 2022.7.3 and are experiencing the same: All gen1 devices (1, 2.5, plug s, 1PM) are not reachable anymore. With the previous release of HA Core 2022.7.2 everything was working smoothly.

HA is also running on Hyper-V ...

Steph-74 commented 2 years ago

Hi @thecode, plesae find attachted my home-assistant.log. Unfortunately, I cannot download any diagnositc data for not available shelly devices (gen1). The option to download diagnostic data is currently available only for gen2 devices.

Kind regards, Stephan Shelly.txt

lanabretterebner commented 2 years ago

Hey @xrsprint @Steph-74 I had exactly the same problems as you guys. I am also running HA on Hyper-V.

I updated to 2022.7.3 and everything worked fine, two days later with no major changes to my config all of a sudden all shellys were unavailable. Unfortunately i do not have a solution to this problem, but i do ave a workaround!

Since Hyper-V is not capable of USB passthrough I have another HA Instance in Oracle Virtual Box VM which is responsible for my Z Wave network. I added the shelly integration to my 2nd instance and connected both instances via Remote Home Assistant.

You can install it via HACS or just follow the instructions on the page linked. This works good for me and it is not that much work to set up either, maybe 30 minutes if you dont already have a second HA instance up and running.

Hope this workaround will fix your problems until a better solution is found.✌

skoog84 commented 2 years ago

Hi,

I got same issue running HA in Hyper-V on Windows 10.

Can you guys that have problem check if you recently installed some windows updates? My integration failed somewhere during the day and below updates were installed 2022-07-13 according to Windows Update history.

image

After i uninstalled them everything worked as before.

Steph-74 commented 2 years ago

Hi @skoog84, I am running on Win 11. Updated to HA 2022.7.3 today and then my shellys gen1 quit working. However, I just checked my Windows update history, I got yesterday following update installed: KB5015814.

After un-installing this windows update my shellys are now connecting to Home Assistant again ... Thanks for the hint.

Hopefully, this gets fixed in HA in an upcoming release.

Best Stephan

xrsprint commented 2 years ago

Hi Everyone i have some updates

@thecode please find attached my logs, i have never had an issue with hyperV until now, so maybe i will look at going back to virtualbox as i was running that some time ago, attached is the log file

home-assistant.zip

@georgbretterebner thanks for that, i was running Virtualbox but found hyper-v much easier, after getting rid of all z-wave devices i thought id give HyperV a go, but may look going back

@skoog84 & @Steph-74 thanks and i have found that yesterday i installed KB5015811 mine might be different as I'm running server 2019. after uninstalling that now all the shelly devices are back online PHEW ! @thecode i looked at the KB details and it does mention service stack updates, but what exactly i don't know and it doesn't mention anything Hyper-V related. for now i wont do any updates, but if there is anything you would like me to try im more than happy to be the test if needed. if there is nothing that can be found, i might revert back to Virtualbox

Thanks again for everyone's input as this morning was a nightmare to turn all lights on and off and arm the alarm :P but will definitely be watching this to see if any progress does get made

Thanks

Eirik78 commented 2 years ago

I also experienced this issue after installing latest KB. Uninstalled it and Shelly was connected again. I see there was a role change for hyperV in that update, but have not looked into the changes.

paolo987 commented 2 years ago

I have the same problem with a HA VM with Shelly devices since yesterday I installed KB5014797 on Hyper-V 2019 version 10.0.17763.3113.

I moved the HA VM to a Hyper-V host where I have NOT installed KB5014797 yet and the Shelly devices are working again.

Should we expect an update from Microsoft or HA or Shelly integration?

HoriCoX24 commented 2 years ago

Hello everyone!

I experienced the same issue since yesterday and thanks to you i finally fixed it! I am running HA on hyper V on a Windows 10 mini-PC and the answer was UNINSTALLING THE KB5015807 as @skoog84 suggested. Thank you so much for everything!

Madness1073 commented 2 years ago

German: July 12, 2022—KB5015814 (OS Build 22000.795 auf dem HOST der VM Maschine deinstallieren - dann geht es wieder Thank you

aerikers commented 2 years ago

Same problem for me. Uninstalled update KB5015811 and I'm back in business. Is this a HA or MS problem?

elupus commented 2 years ago

Absolutely a hyper-v problem. Or a network adapter driver problem.

HoriCoX24 commented 2 years ago

After a quick glance over the change log of the KB5015807 update, you can see that this update was meant to fix some networking issues, fix some group policies and certificates... i don't really get it, but it is what it is.

uspino2 commented 2 years ago

Same thing here. I wonder if we loose something by uninstalling that security patch. Also, will it be reinstalled on the next cumulative update?

bwestphal74 commented 2 years ago

You're all awesome people! I just spent about 8 hours trying everything I could think of to fix this issue... and I just came across this thread, and it fixed the problem. THANK YOU ALL :)

cmilanf commented 2 years ago

I confirm that the issue also happens in Hyper-V from Windows Server 2022 with KB5015827. After uninstalling the update, the integration works again.

dlxpbva commented 2 years ago

Same issue here. Windows 10 Pro 21H2 (10.0.19044 Build 19044) running HAOS in Hyper-V.

KB5015807 breaks the integration. It shows now red exclamation mark next to each Shelly device with "Retrying setup: Timeout during device setup" message.

Removing KB5015807 fixes the problem.

B4dC4rd commented 2 years ago

Wow, this post saved my marriage, big thanks to all of you. I'm glad I did not have to go trough all the troubleshooting to finally get to this root cause. Hopefully this issue will get a permanent fix soon.

elupus commented 2 years ago

Have any of you reported to microsoft?

zc711 commented 2 years ago

Removing a security patch is never a good idea. Maybe HA is doing something not correct. Other Shelly devices work as they should. In my case...

tfyoung commented 2 years ago

I've never looked into how shelly devices work prior to know since it always just worked, but it seems that gen 1 devices use multicast udp/CoAP protocol. At a guess something has gone wrong with multicast subscribe/broadcast with the latest windows update for the virtual switch connecting VMs. There seems to be an option on shelly devices (Under advanced options) to change the multicast into a unicast. That might work for some people who don't need multicast and don't mind entering that on their devices (I imagine you'd set it to \<hass ip>:5683, but haven't tested it).

I don't have any time to get into this at the moment so am just living with the update removed, but If i get time i might re-install it and try and figure out where the multicast is going wrong.

HoriCoX24 commented 2 years ago

I've never looked into how shelly devices work prior to know since it always just worked, but it seems that gen 1 devices use multicast udp/CoAP protocol. At a guess something has gone wrong with multicast subscribe/broadcast with the latest windows update for the virtual switch connecting VMs. There seems to be an option on shelly devices (Under advanced options) to change the multicast into a unicast. That might work for some people who don't need multicast and don't mind entering that on their devices (I imagine you'd set it to :5683, but haven't tested it).

I don't have any time to get into this at the moment so am just living with the update removed, but If i get time i might re-install it and try and figure out where the multicast is going wrong.

I already tried this method, but it did not work :(

cmilanf commented 2 years ago

I've never looked into how shelly devices work prior to know since it always just worked, but it seems that gen 1 devices use multicast udp/CoAP protocol. At a guess something has gone wrong with multicast subscribe/broadcast with the latest windows update for the virtual switch connecting VMs. There seems to be an option on shelly devices (Under advanced options) to change the multicast into a unicast. That might work for some people who don't need multicast and don't mind entering that on their devices (I imagine you'd set it to :5683, but haven't tested it).

In the same fashion as @HoriCoX24, I tried this one as well and it didn’t work; so switching to unicast won’t help with whatever problem the KB introduced.

t3lmoazevedo commented 2 years ago

Same here, UNINSTALLED the KB5015807 as @skoog84 suggested and its WORKING fine now :) Thank to all the community!

FFalcone1 commented 2 years ago

Installed on HA 2022.6 in Hyper-V and Shelly works. With HA 2022.7 it does not work

cmilanf commented 2 years ago

Installed on HA 2022.6 in Hyper-V and Shelly works. With HA 2022.7 it does not work

I tried to revert back to HA 2022.6 and the problem persist as long as the KB5015827 is installed.

cmilanf commented 2 years ago

As I suspected the problem with the 2022-07 Cumulative Update was related with blocking multicast/unicast UDP communications used by CoIoT protocol. So there is a workaround to get your devices data back to Home Assistant, that implies switching to MQTT communication.

I performed the following steps:

  1. I didn't have a MQTT broker, so I installed the Mosquitto Home Assistant Add-on and configured it following documentation.
  2. In you Shelly device administration, untick Enable CoIoT and tick Enable MQTT. Do not use a custom MQTT prefix. A device restart will be needed.
  3. Core Shelly integration only works, as far as I know, with CoIoT protocol, so I removed the integration and setup the ShellyForHASS custom one. Remember that custom integrations are not supported by Home Assistant team, use them at your own risk.
  4. My Shelly devices are detected via MQTT, playing nice with the 2022-07 Cumulative Update of Windows.

Please, notice this is is a workaround using -at your own risk- a custom integration. Ideal scenario would be fix the problem so we can use CoIoT again, but it's not clear whatever this is on the Microsoft or HA side.

tfyoung commented 2 years ago

So the update auto applied again because I wasn't smart enough to stop that, but I've done some testing and my conclusion is that both the linux VMs I have running on hyper-v after the update can't send UDP packets with a payload smaller than 12 bytes.

Anything with a payload of 12 bytes or larger is no problem. Anything 11 or under is off in to the void

The initial UDP packet aioshelly sends is 11 bytes.

So this definitely isn't an aioshelly/ha/shelly integration issue as I can replicate this with just simple python sockets, but it is an issue that's affecting us all.

thecode commented 2 years ago

So the update auto applied again because I wasn't smart enough to stop that, but I've done some testing and my conclusion is that both the linux VMs I have running on hyper-v after the update can't send UDP packets with a payload smaller than 12 bytes.

Anything with a payload of 12 bytes or larger is no problem. Anything 11 or under is off in to the void

The initial UDP packet aioshelly sends is 11 bytes.

So this definitely isn't an aioshelly/ha/shelly integration issue as I can replicate this with just simple python sockets, but it is an issue that's affecting us all.

This is great finding. I setup Hyper-V VM today to debug this and indeed found out that UDP packets are not sent out, but did not dig into trying various packet sizes. With this information it will be probably possible to pad the packets to avoid this I will discuss it with the team members and provide feedback.

meierthomas commented 2 years ago

I'm rebuilding HA (core-2022.7.5) directly onto a x86-64 system and I can't get my Shelly Dimmer 2 (latest firmware) discovered as well. ColoT enabled and unicast selected.

Adding the integration to a brand new system and after submitting the IP address get the 'failed to connect' error.

Last message in the log is Sending request 'cit/d' to device 192.168.2.89 failed_to_connect.txt

FYI, my main setup is a HA (core-2022.7.5) VM running on KVM (Fedora Server 36) but I had the exact same issue with the Shelly integration which is what prompted me to test a NUC installation to take networking etc. out of the equation...

thecode commented 2 years ago

I'm rebuilding HA (core-2022.7.5) directly onto a x86-64 system and I can't get my Shelly Dimmer 2 (latest firmware) FYI, my main setup is a HA (core-2022.7.5) VM running on KVM (Fedora Server 36) but I had the exact same issue with the

Please create a sperate issue. Thanks

meierthomas commented 2 years ago

I'm rebuilding HA (core-2022.7.5) directly onto a x86-64 system and I can't get my Shelly Dimmer 2 (latest firmware) FYI, my main setup is a HA (core-2022.7.5) VM running on KVM (Fedora Server 36) but I had the exact same issue with the

Please create a sperate issue. Thanks

Ok, done. I've added it here as I believe it could be the same issue and therefore not creating additional workload...

thecode commented 2 years ago

I can confirm that padding the external UDP request with a zero fixes the problem

mtomana commented 2 years ago

Ok, done. I've added it here as I believe it could be the same issue and therefore not creating additional workload...

Where is the other issue? Can you please link it here?

meierthomas commented 2 years ago

Ok, done. I've added it here as I believe it could be the same issue and therefore not creating additional workload...

Where is the other issue? Can you please link it here?

I thought what I was experiencing with my problem was similar to what was reported on this issue 75118 but since I was advised to raise a new issue I have done that 75355

n3de commented 2 years ago

What need to be uninstalled? KB5015807, KB5015827, KB5015814, or KB5015811? And how did you uninstall it?

mtomana commented 2 years ago

I'm unable to uninstall this KB because "Security Update for Microsoft Windows (KB5015807) is required by your computer and cannot be uninstalled" :( I tried restarting in safe mode and uninstalling KB from command line (wusa) with no luck... Any ideas?

glasswire17 commented 2 years ago

I've been trying to troubleshoot this issue for days before I came across this article! Uninstalled KB5015814 from my Windows 11 machine and my Shelly 1 is working again!

n3de commented 2 years ago

I've been trying to troubleshoot this issue for days before I came across this article! Uninstalled KB5015814 from my Windows 11 machine and my Shelly 1 is working again!

How do you uninstalled it?

esackbauer commented 2 years ago

@thecode any rough estimate when this could be discussed? I have quite a lot shellies which aren't working now, and uninstalling the security update makes me feel uncomfortable ;)

mtomana commented 2 years ago

I've been trying to troubleshoot this issue for days before I came across this article! Uninstalled KB5015814 from my Windows 11 machine and my Shelly 1 is working again!

How do you uninstalled it?

You can use the command: wusa /uninstall /KB:5015807

ltrienes commented 2 years ago

Experienced the same problem during the initial setup of my 1PM devices, running HA 2022.7.5 in Hyper-v on Windows 10. Uninstalling the KB5015807 update and rebooting Windows fix it.

thecode commented 2 years ago

@thecode any rough estimate when this could be discussed? I have quite a lot shellies which aren't working now, and uninstalling the security update makes me feel uncomfortable ;)

Currently there is no agreement among the team members for patching Shelly to workaround this bug as it is not affecting only Shelly.

thecode commented 2 years ago

@esackbauer had some internal discussion, I am going to verify it doesn't have negative effects on the Gen1 devices (I tested only on two types of devices, going to test with most of them), If it pass I will make this change so we won't need to wait for MS fix.

tfyoung commented 2 years ago

Thanks for working around this bug in MS's code even though it was only affecting a few of us. I certainly appreciate it!

Cheers, Tom

On Thu, 21 Jul 2022 at 03:07, Shay Levy @.***> wrote:

@esackbauer https://github.com/esackbauer had some internal discussion, I am going to verify it doesn't have negative effects on the Gen1 devices (I tested only on two types of devices, going to test with most of them), If it pass I will make this change so we won't need to wait for MS fix.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/75118#issuecomment-1190534118, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACF6XEFFPOADF5KDOEHCE7DVVAW3VANCNFSM53OG7YFQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

paolo987 commented 2 years ago

I have now upgraded to HA Core 2022.7.6 and everything works fine, even the Shelly Gen1s. Thank you very much

xrsprint commented 2 years ago

@thecode thankyou very much for your efforts on this, just updated to latest HA version, updated windows and all good, you guys are the best !!