The RFXCOM RFXtrx integration can no longer start

VorpalBlade commented 6 months ago

The problem

As of yesterday it appears that "RFXCOM RFXtrx" can no longer start.

Stupidly the first thing I tried doing was upgrading HA when I had the issue, I believe I was on 2024.2.1 though, and the upgrade didn't help. But it had worked just one day before. I don't quite see what could be broken to cause this though.

In the logs I have this entry:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/site-packages/RFXtrx/__init__.py", line 990, in _connect
    self.set_recmodes(self._modes)
  File "/usr/local/lib/python3.12/site-packages/RFXtrx/__init__.py", line 1028, in set_recmodes
    data[5] = self._status.device.tranceiver_type
              ^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'device'

What version of Home Assistant Core has the issue?

core-2024.2.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

RFXCOM RFXtrx

Link to integration documentation on our website

https://www.home-assistant.io/integrations/rfxtrx

Diagnostics information

home-assistant_rfxtrx_2024-02-23T20-21-02.816Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

I'm running using docker on a Raspberry Pi 5 8 GB model (that is also used for other things, thus no Home Assistent OS). The zigbee device forwarding works perfectly, it is just the rfxcom one that is broken. So I don't believe it is an issue with access to device files.

version: '3'
services:
  homeassistant:
    container_name: homeassistant
    image: "ghcr.io/home-assistant/home-assistant:stable"
    privileged: true
    environment:
      DISABLE_JEMALLOC: true
    volumes:
      - /home/hass/hass/config:/config
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    network_mode: host
    devices:
      - /dev/serial/by-id/usb-RFXCOM_RFXtrx433_08WD7JBA-if00-port0:/dev/ttyUSB0:rw
      - /dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_b8f625b5aa9aed11b0487808a8669f5d-if00-port0:/dev/ttyUSB1:rw
    group_add:
      - dialout

Additional information

No response

home-assistant[bot] commented 6 months ago

Hey there @danielhiversen, @elupus, @robbie1221, mind taking a look at this issue as it has been labeled with an integration (rfxtrx) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `rfxtrx` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign rfxtrx` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

rfxtrx documentation rfxtrx source _{^{(message by IssueLinks)}}

VorpalBlade commented 6 months ago

Oh this is interesting. For some reason in ttyUSB0 is listed for the ZHA integration as well. I did not set that. Seems like the blame might be on that taking over things it shouldn't. Can't where in the GUI to fix that though.

$ grep -RC 10 ttyUSB0 config/
grep: config/.storage/auth: Permission denied
config/.storage/core.config_entries-      },
config/.storage/core.config_entries-      {
config/.storage/core.config_entries-        "entry_id": "cb76a54b7f2a0af1cd8d3bf269e7d477",
config/.storage/core.config_entries-        "version": 1,
config/.storage/core.config_entries-        "minor_version": 1,
config/.storage/core.config_entries-        "domain": "rfxtrx",
config/.storage/core.config_entries-        "title": "RFXTRX",
config/.storage/core.config_entries-        "data": {
config/.storage/core.config_entries-          "host": null,
config/.storage/core.config_entries-          "port": null,
config/.storage/core.config_entries:          "device": "/dev/ttyUSB0",
config/.storage/core.config_entries-          "automatic_add": false,
config/.storage/core.config_entries-          "devices": {
config/.storage/core.config_entries-            "0a520800040100c52d0149": {
config/.storage/core.config_entries-              "device_id": [
config/.storage/core.config_entries-                "52",
config/.storage/core.config_entries-                "8",
config/.storage/core.config_entries-                "04:01"
config/.storage/core.config_entries-              ]
config/.storage/core.config_entries-            },
config/.storage/core.config_entries-            "0b110001008d2aa201010f60": {
--
config/.storage/core.config_entries-        "unique_id": "15.2151181_59.2747287",
config/.storage/core.config_entries-        "disabled_by": null
config/.storage/core.config_entries-      },
config/.storage/core.config_entries-      {
config/.storage/core.config_entries-        "entry_id": "fe0f2ff35be2bc41e357a062384be297",
config/.storage/core.config_entries-        "version": 1,
config/.storage/core.config_entries-        "minor_version": 1,
config/.storage/core.config_entries-        "domain": "homeassistant_sky_connect",
config/.storage/core.config_entries-        "title": "Home Assistant SkyConnect",
config/.storage/core.config_entries-        "data": {
config/.storage/core.config_entries:          "device": "/dev/ttyUSB0",
config/.storage/core.config_entries-          "vid": "10C4",
config/.storage/core.config_entries-          "pid": "EA60",
config/.storage/core.config_entries-          "serial_number": "b8f625b5aa9aed11b0487808a8669f5d",
config/.storage/core.config_entries-          "manufacturer": "Nabu Casa",
config/.storage/core.config_entries-          "description": "SkyConnect v1.0"
config/.storage/core.config_entries-        },
config/.storage/core.config_entries-        "options": {},
config/.storage/core.config_entries-        "pref_disable_new_entities": false,
config/.storage/core.config_entries-        "pref_disable_polling": false,
config/.storage/core.config_entries-        "source": "usb",
--
config/.storage/core.config_entries-        "disabled_by": null
config/.storage/core.config_entries-      },
config/.storage/core.config_entries-      {
config/.storage/core.config_entries-        "entry_id": "944007ec0cdbc6c19a305a7ae5618af4",
config/.storage/core.config_entries-        "version": 4,
config/.storage/core.config_entries-        "minor_version": 1,
config/.storage/core.config_entries-        "domain": "zha",
config/.storage/core.config_entries-        "title": "SkyConnect v1.0",
config/.storage/core.config_entries-        "data": {
config/.storage/core.config_entries-          "device": {
config/.storage/core.config_entries:            "path": "/dev/ttyUSB0",
config/.storage/core.config_entries-            "flow_control": null,
config/.storage/core.config_entries-            "baudrate": 115200
config/.storage/core.config_entries-          },
config/.storage/core.config_entries-          "radio_type": "ezsp"
config/.storage/core.config_entries-        },
config/.storage/core.config_entries-        "options": {},
config/.storage/core.config_entries-        "pref_disable_new_entities": false,
config/.storage/core.config_entries-        "pref_disable_polling": false,
config/.storage/core.config_entries-        "source": "usb",
config/.storage/core.config_entries-        "unique_id": "10C4:EA60_b8f625b5aa9aed11b0487808a8669f5d_Nabu Casa_SkyConnect v1.0",

VorpalBlade commented 6 months ago

Disabling ZHA doesn't seem to fix it though, but the error is different:

2024-02-23 21:33:25.993 DEBUG (SyncWorker_2) [homeassistant.components.rfxtrx] Using modes: ac,arc,lighting4,hideki
2024-02-23 21:33:25.997 DEBUG (Thread-2 (_connect)) [RFXtrx] Send: 0x0d 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
2024-02-23 21:33:26.300 DEBUG (Thread-2 (_connect)) [RFXtrx] Send: 0x0d 0x00 0x00 0x01 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
2024-02-23 21:33:55.994 ERROR (MainThread) [homeassistant.components.rfxtrx] Connection timeout: failed to receive response from RFXtrx device

VorpalBlade commented 6 months ago

I hooked the rfxcom RFXtrx443 up to a windows virtualbox (I only run Linux natively) and confirmed using RFXmngr that the device itself works perfectly fine.

VorpalBlade commented 6 months ago

By switching to unprivileged podman-compose I managed to get rfxcom working. Can't have both rfxcom and zha working at once though it appears. I'm now using the following compose file:

version: '3'
services:
  homeassistant:
    container_name: homeassistant
    image: "ghcr.io/home-assistant/home-assistant:stable"
    #privileged: true
    environment:
      DISABLE_JEMALLOC: true
    volumes:
      - /home/hass/hass/config:/config
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    network_mode: host
    devices:
      - /dev/serial/by-id/usb-RFXCOM_RFXtrx433_08WD7JBA-if00-port0:/dev/ttyUSB0:rw
    #  - /dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_b8f625b5aa9aed11b0487808a8669f5d-if00-port0:/dev/ttyUSB1:rw
    group_add:
      #- dialout
      - keep-groups

If I uncomment the SkyConnect, RFXCom stops working. Even if I don't re-add the ZHA integration (yes I deleted it, I have way fewer such devices than I have 433 MHz ones still).

I will switch from ZHA to Zigbee2MQTT. And use podman instead of docker I guess. Whatever works...

EDIT: Oh if I remove that restart policy, serial devices stop working under podman. Very stable....

home-assistant[bot] commented 6 months ago

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `zha` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zha` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

zha documentation zha source _{^{(message by IssueLinks)}}

elupus commented 6 months ago

I wonder if zha (or some other part) scans all uarts.

VorpalBlade commented 6 months ago

That sounds likely. At this point I kind of just want to go back to domoticz. It didn't have many features but it ran stably for years. This is the third time in as many months since I switched from domoticz to hass that I have had weird issues. The first two times I managed to solve them myself though.

I switched since I wanted to get into zigbee (rather than just 433) and thought I might as well switch to the modern popular alternative while I was at it.

elupus commented 6 months ago

Well. You do make it hard on yourself with using docker instead of HAOS..

puddly commented 6 months ago

Did you previously have your port mapping defined otherwise? ZHA doesn't change its configuration automatically and the SkyConnect integration filters by PID, VID, manufacturer, model, and serial number. For it to also be pointing to /dev/ttyUSB0 is strange.

VorpalBlade commented 6 months ago

Well. You do make it hard on yourself with using docker instead of HAOS..

I wouldn't mind running hass un-contained as a matter of fact. If I could install it with apt or even just into a home directory and run it from there. The thing is, that Pi is a server for many other things. I don't have hundreds of devices to manage so I can't justify the electricity nor expense of running multiple Pis (electricity isn't cheap here). I wouldn't mind running HASS OS in a VM on the Pi though. But that seemed to only be supported on x86-64?

Did you previously have your port mapping defined otherwise?

Nope, had that mapping in the podman file from the start. Added the SkyConnect device in January. The RFXCOM I added when I switched over from domoticz in early December.

The only recent thing I have done (but not immediately before it broke, a few days before) was update the RPi 5 firmware via rpi-eeprom-update. I can't see how that would be related though.

What does look strange right now is that the mapping on the host has changed (but I restarted a couple of times when troubeshooting, so...), but that shouldn't be related to the device mapping inside the docker container. That is the whole point of the docker container mapping things!

$ ls -l /dev/serial/by-id/
total 0
lrwxrwxrwx 1 root root 13 23 feb 20.17 usb-Nabu_Casa_SkyConnect_v1.0_b8f625b5aa9aed11b0487808a8669f5d-if00-port0 -> ../../ttyUSB0
lrwxrwxrwx 1 root root 13 23 feb 21.44 usb-RFXCOM_RFXtrx433_08WD7JBA-if00-port0 -> ../../ttyUSB1

I wonder if that reversal of the host IDs somehow leaked into the docker container? If so that seems really brittle.

VorpalBlade commented 6 months ago

Hm, would the mapping in the compose file perhaps not be re-evaluated on reboot, causing it to cross over the two devices somehow?

puddly commented 6 months ago

I think providing both privileged and a manual port mapping is going to cause issues.

Reconfigure ZHA to point to the correct serial port either by using the radio migration feature and selecting "re-configure the current radio".

VorpalBlade commented 6 months ago

Aha, they do indeed seem to interact. Interesting. Since I only have two zigbee devices so far (but many 433 MHz ones still) I ended up just deleting the zha integration already before you answered.

Now, I believe I need to run the container privileged according to the official documentation: https://www.home-assistant.io/installation/linux#docker-compose

This seems to be needed for supporting DHCP discovery (I'm getting warnings about that not working now that I'm not running with privileged, though I don't know this is a feature I need).

What is the recommended way to make multiple serial devices work when you use privileged?

aikrana commented 4 months ago

Same problem here with same logs in hassOS on RPI4. I can only start RFXtrx if i disable first ZHA (which is configured to use the serial by-id). Any workarround to solve it? Im currently just starting manually zha after every boot.

` ~ ls -l /dev/serial/by-id total 0

usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_205757d19d29ec1190a8757840c9ce8d-if00-port0 -> ../../ttyUSB1 usb-RFXCOM_RFXtrx433_A18RHEE-if00-port0 -> ../../ttyUSB0 `

VorpalBlade commented 4 months ago

Any workarround to solve it? Im currently just starting manually zha after every boot.

@aikrana So I did a number of things to prevent this from happening again:

I use rootless podman. I don't use HassOS. I set up the devices to use aliases like so:

    devices:
      - /dev/serial/by-id/usb-RFXCOM_RFXtrx433_your_serial_here-if00-poirt0:/dev/ttyRFX:rw
      - /dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_some_random_id-if00-port0:/dev/ttySky:rw

I then use these two aliases /dev/ttyRFX and /dev/ttySky in in HA. That way neither has a "standard" tty name

Now, since you use HassOS I'm not sure how that works, does it still use docker or podman? If not, you may have to do this using udev rules instead, to add more aliases. Or perhaps you can force it to use the paths under /dev/serial directory (more on that later).

I then switched to Zighbee2MQTT instead of ZHA. This uses a separate container, so I no longer have to worry about this happening again at all. I didn't have many zigbee devices (yet at that point) so that wasn't a big deal. Now I have a bunch more and I would not be happy with switching again.
I did need to edit (while hass was stopped!!!) the hass config files to fix up bad paths in the files. Do this at your own risk and make a backup first. You can search for all mentions of ttyUSB in the config directory (assuming that is the name you have it at):

grep -R ttyUSB config/

Path adjustments will presumably be needed for HassOS. No idea how to fully stop hass on HassOS either.

issue-triage-workflows[bot] commented 1 month ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

home-assistant / core