The Tradfri integration stops working every few days.

rubenbe commented 3 years ago

The problem

The Tradfri integration stops working every few days. A restart of the integration (or hass entirely) is required to revive it. No need to restart the ikea tradfri hub.

Environment

Home Assistant Core release with the issue: 0.116.4
Last working Home Assistant Core release (if known): 0.114.x
Operating environment (OS/Container/Supervised/Core): Container on x86
Integration causing this issue: tradfri
Link to integration documentation on our website: https://www.home-assistant.io/integrations/tradfri/

Problem-relevant `configuration.yaml`

Configured via flow

Traceback/Error logs

2020-10-28 17:05:08 ERROR (MainThread) [homeassistant.core] Error executing service: <ServiceCall light.turn_on (c:081e8aff193811ebac0de104ad2b18e1): entity_id=['light.tradfri_bulb_e27_ws_opal_980lm'], params=brightness=175, color_temp=370>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1351, in catch_exceptions
    await coro_or_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1370, in _execute_service
    await handler.func(service_call)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
    await self.hass.helpers.service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 470, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 655, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 507, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 249, in async_handle_light_on_service
    await light.async_turn_on(**params)
  File "/usr/src/homeassistant/homeassistant/components/tradfri/light.py", line 262, in async_turn_on
    await self._api(command)
  File "/usr/src/homeassistant/homeassistant/components/tradfri/base_class.py", line 22, in wrapper
    await func(command)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 164, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 148, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 731, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 783, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 313, in send
    request_interface.request(plumbing_request)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 243, in request
    self.outgoing_requests[key] = request
TypeError: 'NoneType' object does not support item assignment
2020-10-28 17:05:08 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command put ['15001', 65544]: {'3311': [{'5711': 370}]}>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.1.3:5684/15001/65544'))
2020-10-28 17:05:08 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command put ['15001', 65544]: {'3311': [{'5851': 175}]}>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.1.3:5684/15001/65544'))
2020-10-28 17:05:08 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command put ['15001', 65538]: {'3311': [{'5850': 1}]}>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.1.3:5684/15001/65538'))
2020-10-28 17:13:12 ERROR (MainThread) [coap] Connection loss was not expected.

Additional information

Originally discussed here: https://community.home-assistant.io/t/tradfri-integration-stops-working-approx-once-twice-a-week/238429/4 I'll see if the issue keeps appearing on the 0.117.x

rubenbe commented 3 years ago

@Sebazzz

Yes, you first need to open the hass OS linux shell (type login in the HA shell).

Verify you have aiocoap 0.4b3 installed:

docker exec -it homeassistant pip list | grep aiocoap

You can update the version using:

docker exec -it homeassistant pip install aiocoap==0.4.1

After verifying that the update has been correctly installed, restart home-assistant (or the entire OS).

manuelbernhardt commented 3 years ago

@rubenbe thanks! This seems to have gotten the tradfri integration to start again for me after upgrading HA core to 2021.6.3

chriscolden commented 3 years ago

hi all, updated to 2021.6.4. Then upgraded to aiocoap 0.4.1. Tradfri loaded first time after the aiocoap upgrade. :) Thank you so much for your help with this.

chriscolden commented 3 years ago

Started getting this now... Retrying setup: ('There was an error with the request.', NetworkError('[Errno 110] Operation timed out'))

aiocoap 0.4.1

EDIT: Restarting the gateway and reloading the integration solves the problem. Gateway was still working in official ikea smart home app and in google assistant. Which leads me to believe its something to do with this integration and the gateway refusing the connection.

cklam2 commented 3 years ago

Had that same error (Errno 110) yesterday with aiocoap 0.4.1. Worked after restarting gateway but today seeing same problem again.

Before I installed 0.4.1 the integration was just showing "not loaded"

LukasQ commented 3 years ago

Seems like it's broken again. Working:

GW1.16.2, Core: 2021.7.4, OS: Home Assistant OS 6.1

Cracked:

GW: 1.17.1, Core 2021.8.4, OS: Home Assistant OS 6.2.

Unfortunately I realised the cracked integration after all 3 systems updated (automatically), I'll check the rollback scenarios.

rubenbe commented 3 years ago

@Sebazzz

Yes, you first need to open the hass OS linux shell (type login in the HA shell).

Verify you have aiocoap 0.4b3 installed:
docker exec -it homeassistant pip list | grep aiocoap
You can update the version using:
docker exec -it homeassistant pip install aiocoap==0.4.1
After verifying that the update has been correctly installed, restart home-assistant (or the entire OS).

@LukasQ It was broken om my setup too, but this update fixes/works around the problem. Please let me know if that worked. Thanks!

LukasQ commented 3 years ago

with 2021.8.x comes 0.4b3 in my case. 0.4.1 installed ~~and working so far!~~ ~~Update: Nope, also cracked. The switches and lights receive the command, but the update back to HA don't work. Received Type.ACK from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7f8687c42a30>, but could not match it to a running exchange.~~ Update: now working again, without any change. Just lots of restarts,

instantchow commented 3 years ago

with 2021.8.x comes 0.4b3 in my case. 0.4.1 installed ~and working so far!~ ~Update: Nope, also cracked. The switches and lights receive the command, but the update back to HA don't work. Received Type.ACK from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7f8687c42a30>, but could not match it to a running exchange.~ Update: now working again, without any change. Just lots of restarts,

So was the 0.4.1 needed?

LukasQ commented 3 years ago

Unfortunately none of them are working here. 0.4b3 just stops working with ´´´There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://´´´ 0.4.1 stops syncing after a while and just freezes the switch status.

ZatarraNL commented 3 years ago

Coming fresh from Domoticz, I installed HASS two days ago. I experience the same issue. Tradfri works for about a day. After this day the intergration needs to be reloaded, after which everything works again.

I'm not sure yet where to find the right logs yet.

My situation: HASS VM on a proxmox machine x86_64 Hardware: two cores, 4gb RAM OS: Home Assistant OS 6.2 Home Assistant core: 2021.8.8

Let me know where to locate the right log-files, and I'll gladly help.

Donkie commented 3 years ago

Having this issue too. Worked fine for a few weeks but after resetting the Trådfri gateway (and maybe also updating the firmware, not sure) this issue started to happen. Reloading the integration makes it work again for a few hours.

Running Home Assistant core-2021.8.8 as a docker container on a Raspberry Pi 3B with raspbian

Update: Manually updated the aiocoap pip package as suggested above and it seems to have resolved the problem. Will update if anything changes.

hakan-theren commented 3 years ago

I have the same issue running homeassistant in a VM. Tested installing aiocoap 0.4.1 as suggested but as reported before i get problem with syncing devices after a while.

ZatarraNL commented 3 years ago

Yesterday, I installed 2021.9.0 succesfully. Unfortunately, this morning the Ikea Tradfri devices didn't respond anymore. So the issue remains in this version of HA. A reload of the integration makes everything works again, but temporary.

LukasQ commented 3 years ago

Funny, I installed 2021.9.1 (still with aiocoap 0.4b3) and it seems to work since this release. 2021.9.2 also no issues. I'll wait with the installation of OS 6.3 and 2021.9.3 till around Tuesday and come back with more feedback.

hakan-theren commented 3 years ago

A thing i noticed is that when the integration is restarted it seams like all the commands buffered/queued up when the integration is not responding and is then transmitted during restart. The whole house starts blinking from when people has tried turning things on or off as soon as I restart the tradfri integration. The same thing happened when i upgraded the OS.

ZatarraNL commented 3 years ago

Funny, I installed 2021.9.1 (still with aiocoap 0.4b3) and it seems to work since this release. 2021.9.2 also no issues. I'll wait with the installation of OS 6.3 and 2021.9.3 till around Tuesday and come back write more feedback.

And I can confirm - in contrary to my previous reply - Tradfri seems to keep working. So, maybe it's been solved even so. Thanks to all the developpers.

LukasQ commented 3 years ago

So, it is Tuesday and it did work so far without any log entries. Now updating to 2021.9.4. OS 6.3 seems to be withdrawn?!

LukasQ commented 3 years ago

And its dead. Rolled back to 2021.9.1. Update: Now dead with 2021.9.1, too.

LukasQ commented 3 years ago

https://github.com/home-assistant/core/compare/2021.9.1...2021.9.4 Compared these two versions and ... no idea what changed that affects tradfri, I was under the impression that ikea uses zigbee, so also no impact with z-wave changes. I'll have a look at the Supervisor, too. See: https://github.com/home-assistant/supervisor/compare/2021.08.1...2021.09.0 Release is from 3rd and I didn't note down which working version I was using.

LukasQ commented 3 years ago

core 2021.9.6, supervisor-2021.09.0, OS 6.4 with OVA on ESXi. tradfri:latest. Working. No idea what changed.

BubiBalboa commented 2 years ago

I just want to +1 this issue. Same problem for me, sometimes it's better sometimes it's worse. After the most recent update (core-2021.9.6, supervisor-2021.09.0) the integration stops working at least twice a day. A reload fixes it. It could be worse, at least I don't have to reboot the Raspi3 which takes forever, but it is pretty annoying nonetheless.

riccidan commented 2 years ago

I just want to +1 this issue. Same problem for me, sometimes it's better sometimes it's worse. After the most recent update (core-2021.9.6, supervisor-2021.09.0) the integration stops working at least twice a day. A reload fixes it. It could be worse, at least I don't have to reboot the Raspi3 which takes forever, but it is pretty annoying nonetheless.

I too have this issue, sometimes a couple of times a day and other times not for days.

jkushner90 commented 2 years ago

Have posted this in another issue but pasting here as well as I have been following the different open issues related to this and think I have found a workaround that suits me:

Until this issue is resolved I have resorted to using the homekit controller plugin to control the blinds. I tried pairing directly in ZHA but had many issues with slowness with the blinds. Ended up simply pairing the Tradfi hub via homekit instead. Was able to replicate my issue very easily by unplugging the ikea hub. HA would become unresponsive until I restarted the ikea integration. Just tested and that is still happening but the blinds continue to work via the homekit integration. Sorry if this has been discussed before but hoping it can help others in similar situation. Also would love anyones take on this in regards to downfalls of integrating this way. Thanks!

Sebazzz commented 2 years ago

I suspect it is something in the Raspberry PI / ARM based binary. I recently migrated my Home Assistant instance from a Raspberry PI to a cheap low-powered i3 based (x86-64 obviously) NUC, and I haven't had any issues since.

LukasQ commented 2 years ago

x86-64 on ESXi Hypervisor. Issues with 2021.10.3, where 2021.10.2 is working correctly.

jkushner90 commented 2 years ago

Running mine on a Lenovo m600 mini pc. Using generic / NUC image. Have had this issue since starting with home assistant about 4 months back. Happened with pi3 and now this new hardware. Easy to replicate, just unplug the ikea hubs power for a second and everything connected to it will become unresponsive in home assistant until I restart the ikea integration. Waiting hours does nothing. Running unifi dream machine and both devices are connected to the same switch in my network. Seems to also be doing it just from a disconnection of the ethernet to the ikea hub. Guessing that’s why some people have this issue much more frequent than other; quality of network equipment keeping connection alive is playing large role.

VNRARA commented 2 years ago

I have the same issue but running HA on a synology NAS 24/7...

Strohhutpat commented 2 years ago

issue with 2021.11 still present, have to restart integration within every ~3 hours

st3v3nFr commented 2 years ago

Can't understand why we are still dealing this issue 1 year later. Is there another way of asking support to the team ???

These last couples of days i had to restat Ikea HUB AND HA once a day, this is very annyoing :/

edtro commented 2 years ago

Hi, I also had this issue with the standard Tradfri integration (I had to reset every hour or so). Thanks to @jkushner90 I am now using the HomeKit Controller integration instead for all of my Ikea Tradfri devices (plugs, buttons etc), see: https://www.home-assistant.io/integrations/homekit_controller/

Now it works again as it should.

Please note: I am NOT using HomeKit itself. Just the integration, that is acting like it is an HomeKit instance. Just remove the standard Tradfri integration, restart HASS... and use the HomeKit controller integration (instead of using the default security code, use the HomeKit securtity code: 123-45-678). The devices/entities will be recreated, but I have noticed no changes of the internal unique names within my config (the same names where used). I just had to modify a couple of scripts/automations, because instead of names, guids where used within the yaml scripts to identify the targetted entites. So it was not that hard to do.

Kr Edward

crhuber commented 2 years ago

also having this issue

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/tradfri/base_class.py", line 74, in _async_run_observe
    await self._api(cmd)
  File "/usr/src/homeassistant/homeassistant/components/tradfri/base_class.py", line 39, in wrapper
    await func(command)
  File "/usr/local/lib/python3.9/site-packages/pytradfri/api/aiocoap_api.py", line 196, in request
    self.debug_comm("call", api_commands)
  File "/usr/local/lib/python3.9/site-packages/pytradfri/api/aiocoap_api.py", line 188, in debug_comm
    api_msg += f"<<<{vars(api_command)}>>>"
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 855, in __repr__
    return f"<Entity {self.name}: {self.state}>"
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 902, in state
    return STATE_ON if self.is_on else STATE_OFF
  File "/usr/src/homeassistant/homeassistant/components/tradfri/light.py", line 147, in is_on
    return cast(bool, self._device_data.state)
  File "/usr/local/lib/python3.9/site-packages/pytradfri/device/light.py", line 40, in state
    return self.raw.get(ATTR_DEVICE_STATE) == 1
  File "/usr/local/lib/python3.9/site-packages/pytradfri/device/light.py", line 85, in raw
    return self.device.raw[ATTR_LIGHT_CONTROL][self.index]
KeyError: '3311'

kongjudas commented 2 years ago

It seems to getting worse by the day here, just minutes after a HA reboot. I have had to restart the IKEA integration almost every time I want to switch the light (since I'm not doing it every minute). I also notice that, when it happens, the IKEA remotes stops working as well. These should be unrelated to HA, so it seems something's happening to the Trådfri gateway or mesh when it freezes. Reading forums, I've seen this problem going years back. As the dev. said in the latest HA release party: The least we can expect from an automated home is to be able to switch our light on and off.

nebulorum commented 2 years ago

Not sure this is related, but I reach this issue locking from connection problems with my Tradfri gateway. It works perfectly, until I added some automation on NodeRed. Now every time I run the automation, HA loses the ability to control. Curiously this does not happen for all automations. Turning off a single device is ok. But an automation that dims all devices in a room them restores them is actually causing HA to loose connection. I can recover be reloading the integration.

rubenbe commented 2 years ago

There recently was a new release of the "low level" library that is used to communicate with the gateway. If possible give it a try and report your findings here. I installed it using: docker exec -it homeassistant pip install aiocoap==0.4.3

Mariusthvdb commented 2 years ago

yeah but HA OS users can not can we? No such thing as a CC for that, as its a library used by the pytradfri library which is used by the integration..... all a matter of who is in charge of what ? Personally I am lost in the various PR's/issues, and would hope some central command could take control

AndreasPaulsson commented 2 years ago

I also have this problem, running HA 2021.11.5 on HAOS 5.10.75 on an Intel NUC:

Unable to execute command <Command put ['15001', 65560]: {'3311': [{'5850': 1}]}>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.100.20:5684/15001/65560'))
1 december 2021 17:41:13 – (FEL) IKEA TRÅDFRI - Meddelandet inträffade först 1 december 2021 17:41:13 och har hänt 2 gånger

there is also a log warning at the same second, right before this message which may be related to aiocoap:

Protocol is shutdown, cancelling command: 192.168.100.20 <Command put ['15001', 65564]: {'3311': [{'5851': 200}]}>
1 december 2021 17:41:13 – (VARNING) /usr/local/lib/python3.9/site-packages/pytradfri/api/aiocoap_api.py - Meddelandet inträffade först 1 december 2021 17:41:13 och har hänt 4 gånger

It has happened a few times the last month but it feels like it is happening more and more often now. Usually when it happens, I can connect to the IKEA hub with the IKEA app on iOS, but on one occasion, the Tradfri hub needed a reboot, so it may be related to problems with the Tradfri hub as well.

Mariusthvdb commented 2 years ago

those message are from after the reload probably? at least thats what I see in my stack-trace upon auto reload.

I do experience the need for a Gateway restart every now and then too, but as far as I can analyze that, it is because the HA integration (or underlying libraries) cause some sort of traffic jam...

the iOS app and HA integration use completely different communication protocols as I have been explained here (or in other related issues) so the only thing we ca deduct from that is its not the Ikea setup causing trouble . Which is a good thing. (Always also use the proprietary setup. If Ha breaks, you can at least still use the hardware)

pomah86 commented 2 years ago

Any news on this? For me it is really easy to reproduce, disconnect the network cable from the trådfri gateway, connect it again, or just restart the trådfri gateway. Hassio can no longer control the lights, you need to restart the integration for it to work again.

Running the following setup:

maweki commented 2 years ago

you need to restart the integration for it to work again

For me that's not the case with all lights. Some remain stuck with the following error:

Logger: homeassistant.components.light
Source: helpers/entity_platform.py:586
Integration: Licht (documentation, issues)
First occurred: 28. November 2021, 15:20:51 (22 occurrences)
Last logged: 14:37:51

Platform tradfri does not generate unique IDs. ID light-7e4641520440012a-65537 already exists - ignoring light.kuchenschrank

Only restarting hass solves this for me

st3v3nFr commented 2 years ago

Hi everyone. I'm with this issue despite several months, and it was really annoying : most of my lights / switch are from tradffri. After waiting for so long to get any patch, i've moved my 70 things to deconz, hoping i won't have other issues with it.

For today, that was the only thing that could help me : get rid of ikea hub. it's a mess, because it was working fine at the begining.

Hope evryone will have a good ending with this usse !

st3v3nFr commented 2 years ago

you need to restart the integration for it to work again

For me that's not the case with all lights. Some remain stuck with the following error:
Logger: homeassistant.components.light
Source: helpers/entity_platform.py:586
Integration: Licht (documentation, issues)
First occurred: 28. November 2021, 15:20:51 (22 occurrences)
Last logged: 14:37:51

Platform tradfri does not generate unique IDs. ID light-7e4641520440012a-65537 already exists - ignoring light.kuchenschrank
Only restarting hass solves this for me

I have the same as you but for deconz too : i think this more a 'bug' in HA, because i didn't have this one before 2021.10

Mariusthvdb commented 2 years ago

there is serious development in the underlying library https://github.com/home-assistant-libs/pytradfri/pull/392 being merged into dev currently. Lets keep our fingers crossed this will aid to the fix.

thanks to Martin and al who have been busy with the issue!

ChipWolf commented 2 years ago

there is serious development in the underlying library home-assistant-libs/pytradfri#392 being merged into dev currently. Lets keep our fingers crossed this will aid to the fix.

thanks to Martin and al who have been busy with the issue!

Looks merged and tagged (7.2.1) shortly before you posted this, just needs a bump https://github.com/home-assistant/core/blob/06d29040b922890ec2a33faf8cf39c25ad5f71a0/requirements_all.txt#L1969-L1970

Mariusthvdb commented 2 years ago

a right. that explains why things havent improved just yet....? I guess that will be taken care of by Martin soon. thanks for checking!

edit:

no it doesnt, as it was merged and bumped: https://github.com/home-assistant/core/blob/5fd4c374274cae0ecabec045cbae21d79d388241/requirements_all.txt#L1975

at least in dev, and thats what I am running. Issue seems to have intensified as a matter of fact...

Mariusthvdb commented 2 years ago

separate post for a new startup log warning:

Logger: coap
Source: /usr/local/lib/python3.9/site-packages/aiocoap/messagemanager.py:217 
First occurred: 00:50:34 (2 occurrences) 
Last logged: 00:50:34

Received Type.ACK from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7f80f1daf0>, but could not match it to a running exchange.

ChipWolf commented 2 years ago

a right. that explains why things havent improved just yet....? I guess that will be taken care of by Martin soon. thanks for checking!

edit:

no it doesnt, as it was merged and bumped:

https://github.com/home-assistant/core/blob/5fd4c374274cae0ecabec045cbae21d79d388241/requirements_all.txt#L1975

at least in dev, and thats what I am running. Issue seems to have intensified as a matter of fact...

The changes were tagged in a dev core release 11 hours ago @Mariusthvdb https://github.com/home-assistant/core/releases/tag/2021.12.0b2

Mariusthvdb commented 2 years ago

thanks for tagging me, this is/was driving me mad, up to the point I created a new issue only yesterday to get some integration in occurring issues with the Tradfri integration https://github.com/home-assistant/core/issues/61103

Must admit that after having updated to 2022.1.0.dev20211207 this morning (which will probably have the same changes) I haven only seen

2021-12-07 10:47:17 WARNING (MainThread) [coap] Received Type.ACK from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fa0898490>, but could not match it to a running exchange.

not sure what that means, but fingers crossed it will remain at that.... has only been 2 hours now ;-)

Mariusthvdb commented 2 years ago

nope: still all mentioned in #61103 is there

ChipWolf commented 2 years ago

nope: still all mentioned in #61103 is there

Agree, still seeing the same, albeit not as often

home-assistant / core