ZHA fails to load - Retrying setup: [Errno 2] could not open port /dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2134161-if00: [Errno 2] No such file or directory

[jcallaghan]

The problem

ZHA fails to start after upgrading to 2021.05.0 and 2021.05.1. If I revert back to 2021.4.6 ZHA loads fine.

What is version of Home Assistant Core has the issue?

2021.5.1

What was the last working version of Home Assistant Core?

2021.4.6

What type of installation are you running?

Home Assistant Core

Integration causing the issue

ZHA

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Following in the log

Logger: homeassistant.components.zha.core.gateway
Source: components/zha/core/gateway.py:157
Integration: Zigbee Home Automation (documentation, issues)
First occurred: 1:37:09 PM (2 occurrences)
Last logged: 1:37:19 PM

Couldn't start deCONZ = dresden elektronik deCONZ protocol: ConBee I/II, RaspBee I/II coordinator
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/serial/serialposix.py", line 322, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2134161-if00'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 157, in async_initialize
    self.application_controller = await app_controller_cls.new(
  File "/usr/local/lib/python3.8/site-packages/zigpy/application.py", line 69, in new
    await app.startup(auto_form)
  File "/usr/local/lib/python3.8/site-packages/zigpy_deconz/zigbee/application.py", line 65, in startup
    await self._api.connect()
  File "/usr/local/lib/python3.8/site-packages/zigpy_deconz/api.py", line 245, in connect
    self._uart = await zigpy_deconz.uart.connect(self._config, self)
  File "/usr/local/lib/python3.8/site-packages/zigpy_deconz/uart.py", line 142, in connect
    _, protocol = await serial_asyncio.create_serial_connection(
  File "/usr/local/lib/python3.8/site-packages/serial_asyncio/__init__.py", line 445, in create_serial_connection
    serial_instance = serial.serial_for_url(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/serial/__init__.py", line 90, in serial_for_url
    instance.open()
  File "/usr/local/lib/python3.8/site-packages/serial/serialposix.py", line 325, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 2] could not open port /dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2134161-if00: [Errno 2] No such file or directory: '/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2134161-if00'

### Additional information

The following is shown on the integration page.

![image](https://user-images.githubusercontent.com/7449908/117539531-04643a80-b003-11eb-8915-c3d3a689b2ac.png)

[probot-home-assistant[bot]]

Hey there @dmulcahey, @adminiuga, mind taking a look at this issue as its been labeled with an integration (zha) you are listed as a codeowner for? Thanks! _{^{(message by CodeOwnersMention)}}

[Adminiuga]

There's no serial port device file. Zha does not create that file, so the problem is not caused by zha. Check what creates that file (udev), afaik there were reports of udev breaking in the update?

But this is not issue caused by zha

[Adminiuga]

Also, try re-plugging the conbee

[jcallaghan]

Thanks, @Adminiuga. No change when removing and adding the stick. I've reverted to 2021.04.6 as I'm quite dependant on ZHA running particularly with this many devices 🙃

[Adminiuga]

Is it home-assistant core? Or docker? Core on Python venv doesn't do any low level device management and just update shouldn't cause the device to disappear

[jcallaghan]

Apologies yes running in Docker.

[davet2001]

I had the same issue. /dev/serial is not present in the docker container for me, even though it's present in the host OS and in the portainer config file.

[Adminiuga]

New docker images are not running udev daemon anymore? Check the release notes for the docker images

[davet2001]

Maybe this is related? https://github.com/home-assistant/docker/pull/180

[Adminiuga]

Pretty sure it should fix the problem

[davet2001]

Pretty sure it should fix the problem

I was thinking that PR might have introduced the problem.

[jendib]

How are we supposed to map USB devices like ConBee now? (I'm using the Supervised version)

[Adminiuga]

Yes, you are right. That's the breaking change from the release notes https://www.home-assistant.io/blog/2021/05/05/release-20215/#breaking-change

Set the environment variable to enable old behavior. Optionally provide feedback in the docker repo, as the maintainers would like to know how that is being used.

Personally, I'm not fond of running privileged docker, so the options are:

• pass through the device -- but docker doesn't handle hot plugging
• mount the dev folder and use control groups for character devices

ATM i did the later.

[Adminiuga]

The issue is caused by the breaking change home-assistant/docker#180

Setting the environment variable should enable olde behavior. Closing, as this is not a zha issue.

[davet2001]

I tried adding

    environment:
     - USING_UDEV=1

To the end of my stack configuration file. The re-updated the stack. But still no different

bash-5.0# ls /dev/serial
ls: /dev/serial: No such file or directory
bash-5.0# 

All ZHA devices are offline, and I can see /dev/ttyUSB0 etc, but I don't see any way to reconfigure the usb stick to use a different path without deleting the integration. Presumably I have to re-add all devices if I do that.

[Adminiuga]

You could carefully edit .storage/core.config_entries file but removing and adding integration is safer way. But may loose some customizations, haven't tried this in a while

[davet2001]

You could carefully edit .storage/core.config_entries file but removing and adding integration is safer way. But may loose some customizations, haven't tried this in a while

Thanks, editing .storage/core.config_entries worked. Feels like a bit of a hack! For anyone trying to repeat I did:

1. Stop the container.
2. In the host machine terminal run ls -laR /dev/serial to list which physical USB devices the symlinks are mapped to.
3. Make sure that your docker config file maps any usb devices you want to use into the container, e.g.

   ...
   devices:
   - /dev/ttyUSB0:/dev/ttyUSB0
   - /dev/ttyUSB1:/dev/ttyUSB1

4. Edit .homeassistant/.storage/core.config_entries, replacing /dev/serial/by-id/xxxxxyyyyy with the device mapped above e.g. /dev/ttyUSB0
5. Don't edit anything else!
6. Save and close.
7. Start the container.

Now zigbee devices all work again.