Light Switches do not work

[Bolten88]

Since today the switches react very bad or not at all. But i do not see any errors in the log. When i click to turn on a light just nothing is happening. This morning i saw in the IKEA app that there were updates installed. Did this broke the plugin? Sometimes the switch turns on/off when i wait for like 30 seconds.

[basrieter]

I can confirm this issue. Started yesterday after a firmware update. Domoticz logfile does not show any issues.

[martin-g-it]

Same issue over here.. Renewing the keys (python23 plugin.py config IP KEY) did work to re-establish the plugin for nodered, The Domoticz plugin however only allows one (and only) action after which the plugin becomes irresponsive. Restarting the plugin (deactivate/activate via hardware) or restarting the Domotiz service again allows you to perform one single action.

[basrieter]

Same issue over here.. Renewing the keys (python23 plugin.py config IP KEY) did work to re-establish the plugin for nodered, The Domoticz plugin however only allows one (and only) action after which the plugin becomes irresponsive. Restarting the plugin (deactivate/activate via hardware) or restarting the Domotiz service again allows you to perform one single action.

Exactly my experience. If I wait for along time, I can do another command. Seems like after each command there is a time-out or so.

[korvanessen]

Same issue over here.. Renewing the keys (python23 plugin.py config IP KEY) did work to re-establish the plugin for nodered, The Domoticz plugin however only allows one (and only) action after which the plugin becomes irresponsive. Restarting the plugin (deactivate/activate via hardware) or restarting the Domotiz service again allows you to perform one single action.

Same here.

[moroen]

I've noticed this as well, it's possible to send one command, then after a long period, it's possible to send another. It must have something to do with the latest firmware update. I've not been able to research it fully yet, but I suspect that switching to using coapcmd instead of py3coap might solve this, at least as a temporary measure. I'll look further into this as soon as possible.

[Bolten88]

I've noticed this as well, it's possible to send one command, then after a long period, it's possible to send another. It must have something to do with the latest firmware update. I've not been able to research it fully yet, but I suspect that switching to using coapcmd instead of py3coap might solve this, at least as a temporary measure. I'll look further into this as soon as possible.

Thanks for confirming the issue. Keep us posted!

I noticed that 'observing changes' in Domoticz still works fine when you use the IKEA app or remotes.

[moroen]

I've been able to verify that using coapcmd seems to avoid this issue, so switching to coapcmd is a temporary solution:

$ cd ../plugins/IKEA-Tradfri
$ sh ./install_coapcmd.sh
$ python3 plugin.py api coapcmd
$ python3 plugin.py api list

This requires that git and go is installed.

Alternatively, prebuild binaries are available for x86_64, arm 6 and 7, darwin (macos) and windows. Download the correct binary from https://github.com/moroen/coapcmd/releases and put the binary into ./bin in the IKEA-Tradfri directory and rename it to coapcmd (coapcmd.exe on windows).

[martin-g-it]

Indeed.. coapcmd still seems to work.. Is it me or is this way of working less responsive compared to what pycoap used to be? Eg. dimmer level switches back to the previous value after changing to a different level.

Anyway it's a good workaround for now :-) thanks!

[martin-g-it]

Not sure if this is known behavior, but for what's it worth; the python3 plugin.py list and python3 plugin.py raw <ID> commands don't work (anymore) with the coapcmd whereas they still do work with pycoap...

[moroen]

coapcmd should work just as pycoap, both list and raw commands works on my system... Do you get any errors? Perhaps open a separate issue for this?

[mpranger]

I've had the same initial problem. The workaround was OK, but had the same problem that list and raw did not work at first.

How my system responded:

python3 plugin.py list ==> Connection timed out sudo service domoticz restart python3 plugin.py list ==> All OK

All now seems to be OK, although I don't understand what the relation between my action and the change in response is. Hope you get to the root cause :-)

[moroen]

I've had the same initial problem. The workaround was OK, but had the same problem that list and raw did not work at first.

How my system responded:

python3 plugin.py list ==> Connection timed out sudo service domoticz restart python3 plugin.py list ==> All OK

All now seems to be OK, although I don't understand what the relation between my action and the change in response is. Hope you get to the root cause :-)

There shouldn't be any relation, but I'll wager that it's somehow is related to how the gateway handles simultaneous connections...

I'm struggling to determine what the last firmware has changed and thus why pycoap fails. After a request has been sent and handled correctly (the "first" switching of a device), the following requests are simply being ignored for a while. No errors, no timeout, no anything, just ignored. Then after a while, and the length of the intervall is not constant, a request is handled correctly again, before the following requests are ignored again for a while...

coapcmd works because it creates a connection, sends a request and returns the response, and then closes the connection again, creating a new connection for the next request and so on...

I'll continue to look into this and try to either fix this or create a workaround, but this might be a more momentous task than anticipated.

[Bolten88]

I have changed to coapcmd but it is not error free, the behavior looks similar with pycoap but it handles it better. When i do

python3 plugin.py list i get an 'connection time out' but if you keep trying it will be succesfull sometime:

pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Lights: 65537: Driver-Koffiehoek (State: 1 - Level: 63 - Hex: None) 65540: Led-lamp-TV (State: 1 - Level: 203 - Hex: None) 65541: Led-lamp-Eettafel (State: 1 - Level: 203 - Hex: None) Others: 65536: Afstandsbediening-Koffiehoek (State: None - Level: None - Hex: None) 65539: schakelaar-Woonkamer (State: None - Level: None - Hex: None) Batteries: 65536: Afstandsbediening-Koffiehoek - 87 65539: schakelaar-Woonkamer - 60 pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Connection timed out pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py list Lights: 65537: Driver-Koffiehoek (State: 1 - Level: 63 - Hex: None) 65540: Led-lamp-TV (State: 1 - Level: 203 - Hex: None) 65541: Led-lamp-Eettafel (State: 1 - Level: 203 - Hex: None) Others: 65536: Afstandsbediening-Koffiehoek (State: None - Level: None - Hex: None) 65539: schakelaar-Woonkamer (State: None - Level: None - Hex: None) Batteries: 65536: Afstandsbediening-Koffiehoek - 87 65539: schakelaar-Woonkamer - 60 pi@raspberrypi:~/domoticz/plugins/IKEA-Tradfri $

The switches in Domoticz are working again but i agree that it is not that smooth compared to pycoap. Sometimes it takes several seconds before the switch actualy turnes on.

And when i keep the IKEA app next to it happens sometimes that the connection with the app disconnects. Haven't seen thtat with pycoap. Might be an combination with the firmware and coapcmd.

[moroen]

coapcmd is a bit slower than py3coap, but I think the main problem here, is your frequent timeouts, which probably isn't due to the plugin or coapmd. Coapcmd (and py3coap) have, by design, short timeouts. Both coapcmd and py3coap use the same network library (https://github.com/moroen/gocoap), and thus have the same vulnerability for network latency, but py3coap hides and handles timeouts a bit better that coapcmd.

I have noticed though, that on my own setup, there has been an marked increase in timeouts from the gateway after the latest firmware update, nothing else has changed, so I asume this is part of what's changed with the latest firmware.

It's possible to tune coapcmd to allow a bit more latency before timing out, but I'll focus my efforts on py3coap at the moment. If getting py3coap to work fails, and coapcmd becomes the way forward, I'll make it more robust against network latency.

[moroen]

The latest version of pycoap (0.9.0) and the plugin (0.9.5) now works with the latest IKEA Tradfri firmware 1.11.47. It's necessary to update both pycoap and the plugin to the latest version.

I'm not completely satisfied with this solution, it's a brute force solution to a problem I don't fully understand, but at least it's possible to switch more than one device without waiting > 45 seconds before switching another.

Unfortunately, it's noticeably slower when switching multiple devices than the previous version. When switching just one device, there shouldn't be much of a delay, but when switching several devices (as when using scenes), there will be a noticeable delay between lights changing state. On my system this delay is around 1-2 seconds between each light switching, so the lights turn on/off in an obvious sequence.

At the moment this can't be helped. I've not been able to figure out why the gateway just ignores PUT requests for around 45 seconds after receiving a PUT-request. It might be some kind of acknowledgement-issue, but since the API is not public, it's impossible to tell. The plugin and pycoap works around this issue by keeping the connection open between GET requests, but closes it after a PUT-request. The next GET or PUT request will then reopen the connection. Since updating several lights at once, entails several PUT requests, the connection gets closed and reopened several times, hence the delay.

[Bolten88]

The latest version of pycoap (0.9.0) and the plugin (0.9.5) now works with the latest IKEA Tradfri firmware 1.11.47. It's necessary to update both pycoap and the plugin to the latest version.

I'm not completely satisfied with this solution, it's a brute force solution to a problem I don't fully understand, but at least it's possible to switch more than one device without waiting > 45 seconds before switching another.

Unfortunately, it's noticeably slower when switching multiple devices than the previous version. When switching just one device, there shouldn't be much of a delay, but when switching several devices (as when using scenes), there will be a noticeable delay between lights changing state. On my system this delay is around 1-2 seconds between each light switching, so the lights turn on/off in an obvious sequence.

At the moment this can't be helped. I've not been able to figure out why the gateway just ignores PUT requests for around 45 seconds after receiving a PUT-request. It might be some kind of acknowledgement-issue, but since the API is not public, it's impossible to tell. The plugin and pycoap works around this issue by keeping the connection open between GET requests, but closes it after a PUT-request. The next GET or PUT request will then reopen the connection. Since updating several lights at once, entails several PUT requests, the connection gets closed and reopened several times, hence the delay.

That is great news! I updated right away and seems to work again. I will test it better today and let you know. Can you share some technical details about the fix? Just curious.

[Bolten88]

What i did to update all what is neccesary:

sudo pip3 install -U pip --> To update PIP
sudo -H pip3 install -U py3coap --> To update py3coap
cd /home/pi/domoticz/plugins/IKEA-Tradfri/ --> go to plugin directory
git pull --> Update IKEA plugin
python3 plugin.py version --> Check Plugin version
python3 plugin.py api pycoap --> Switch to pycoap
sudo service domoticz.sh restart --> restart domoticz

[basrieter]

Great news! Would it be possible to decompile the IKEA Android app to see how they use the API.

[moroen]

That is great news! I updated right away and seems to work again. I will test it better today and let you know. Can you share some technical details about the fix? Just curious.

It's not as much a fix as it is a workaround. When my own system failed, I discovered that using my command-line tool, it was still possible to switch a device without any pause or to switch one device and then immediately switch another. This led me to conclude that coapcmd probably would work, which it did.

The main difference between pycoap and coapcmd, is that while coapcmd uses a one-off connection to the gateway (connect - authenticate - send request - receive response - close connection), pycoap opens a connection on the first request, and keeps that connection open for the entire lifetime of the plugin (or at least until a request somehow fails, after which it tries to reconnect immediately). This was done to enable peacefull coexistense with Home Assistant, and also to make the plugin more responsive.

Now, after the latest firmware update, using the persistent connection for multiple GET requests (getting the info FROM the gateway) worked as it used to. When using the persistent connection for a PUT request (setting info on the gateway, i.e. to switch a device) the gateway switches the device and returns the expected information the first time. Subsequent PUT-requests are then just ignored for about 45 seconds. No errors, no timeout, the response returned from the gateway has the proper COAP code (changed), but the device is not switched and the info from the gateway shows that the device remains in the old state.

After waiting for about 45 seconds, the persistent connection can be used to send a new PUT-request, which behaves as it should, but then further requests are being ignored for a new period of around 45 seconds... Any PUT requests in this periode are just ignored, while GET requests work as they should.

So to work around this, now the connection is closed after a PUT-request, and remain closed until the next request comes along. If the next request is a PUT request, the connection is closed again after it's done, if the next request is a GET, it's kept open until a PUT-request comes along.

Hopefully it will be possible to determine why a PUT-request is followed by a 45 second pause and fix this, if not I have som ideas on how to perhaps get the responsetime a bit shorter, but as of now, this is as good as it gets...

Regards, M

[Bolten88]

Once in a while my IKEA devices stop to update. I also cannot switch any light then. When i disconnect the connection between domoticz and the IKEA hub it starts to work again. All these steps work: -Reboot domoticz -Reboot IKEA hub -Disconnect network cable from domoticz or IKEA hub

Anything you have seen as well?

[mpranger]

My system does not respond well with pycoap, but it does respond well with coapcmd.

What goes wrong with pycoap:

• Once I switched on/off/changed color a lamp, the lamp becomes unresponsive. Sometimes the response comes back, sometimes not. ==> OK with coapcmd
• With Domoticz events I scheduled a Blocky event that switches on the lights in the morning, changes the color to daylight, and then switches off again (a reset of the warm light of the evening). This morning some lights were still on, some lights were off, some lights were on with warm light. I guess this is the same problem as the unresponsiveness, but it demonstrates how unpredictability. ==> will see tomorrow if this goes OK with coapcmd

Hope this helps in getting a step further.

[moroen]

Once in a while my IKEA devices stop to update. I also cannot switch any light then. When i disconnect the connection between domoticz and the IKEA hub it starts to work again. All these steps work: -Reboot domoticz -Reboot IKEA hub -Disconnect network cable from domoticz or IKEA hub

Anything you have seen as well?

No, my system has been stable since the upgrade. How often does this happen?

[moroen]

My system does not respond well with pycoap, but it does respond well with coapcmd.

What goes wrong with pycoap:

• Once I switched on/off/changed color a lamp, the lamp becomes unresponsive. Sometimes the response comes back, sometimes not. ==> OK with coapcmd
• With Domoticz events I scheduled a Blocky event that switches on the lights in the morning, changes the color to daylight, and then switches off again (a reset of the warm light of the evening). This morning some lights were still on, some lights were off, some lights were on with warm light. I guess this is the same problem as the unresponsiveness, but it demonstrates how unpredictability. ==> will see tomorrow if this goes OK with coapcmd

Hope this helps in getting a step further.

This sound a lot like the original problem, so I have to ask: Are you sure you've updated both pycoap and the plugin to the latest versions?

[mpranger]

My system does not respond well with pycoap, but it does respond well with coapcmd. What goes wrong with pycoap:

• Once I switched on/off/changed color a lamp, the lamp becomes unresponsive. Sometimes the response comes back, sometimes not. ==> OK with coapcmd
• With Domoticz events I scheduled a Blocky event that switches on the lights in the morning, changes the color to daylight, and then switches off again (a reset of the warm light of the evening). This morning some lights were still on, some lights were off, some lights were on with warm light. I guess this is the same problem as the unresponsiveness, but it demonstrates how unpredictability. ==> will see tomorrow if this goes OK with coapcmd

Hope this helps in getting a step further.

This sound a lot like the original problem, so I have to ask: Are you sure you've updated both pycoap and the plugin to the latest versions?

python3 plugin.py version ==> IKEA Tradfri Plugin - version 0.9.5 sudo -H pip3 install -U py3coap ==> Requirement already up-to-date: py3coap in /usr/local/lib/python3.7/dist-packages (0.9.0)

[Bolten88]

Once in a while my IKEA devices stop to update. I also cannot switch any light then. When i disconnect the connection between domoticz and the IKEA hub it starts to work again. All these steps work: -Reboot domoticz -Reboot IKEA hub -Disconnect network cable from domoticz or IKEA hub Anything you have seen as well?

No, my system has been stable since the upgrade. How often does this happen?

It happens like 4 to 5 times a day. When i keep 'playing' with the lights everything is stable. When i leave it for a short while and want to check it later they do not respond anymore. I see this in the log when i try to switch a light then:

2020-09-22 11:33:46.527 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly
2020-09-22 11:34:56.539 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly

[moroen]

My system does not respond well with pycoap, but it does respond well with coapcmd. What goes wrong with pycoap:

• Once I switched on/off/changed color a lamp, the lamp becomes unresponsive. Sometimes the response comes back, sometimes not. ==> OK with coapcmd
• With Domoticz events I scheduled a Blocky event that switches on the lights in the morning, changes the color to daylight, and then switches off again (a reset of the warm light of the evening). This morning some lights were still on, some lights were off, some lights were on with warm light. I guess this is the same problem as the unresponsiveness, but it demonstrates how unpredictability. ==> will see tomorrow if this goes OK with coapcmd

Hope this helps in getting a step further.

This sound a lot like the original problem, so I have to ask: Are you sure you've updated both pycoap and the plugin to the latest versions?

python3 plugin.py version ==> IKEA Tradfri Plugin - version 0.9.5 sudo -H pip3 install -U py3coap ==> Requirement already up-to-date: py3coap in /usr/local/lib/python3.7/dist-packages (0.9.0)

Just comitted a small patch, please try upgrading the plugin to version 0.9.6

[moroen]

Once in a while my IKEA devices stop to update. I also cannot switch any light then. When i disconnect the connection between domoticz and the IKEA hub it starts to work again. All these steps work: -Reboot domoticz -Reboot IKEA hub -Disconnect network cable from domoticz or IKEA hub Anything you have seen as well?

No, my system has been stable since the upgrade. How often does this happen?

It happens like 4 to 5 times a day. When i keep 'playing' with the lights everything is stable. When i leave it for a short while and want to check it later they do not respond anymore. I see this in the log when i try to switch a light then:

2020-09-22 11:33:46.527 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly
2020-09-22 11:34:56.539 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly

Some kind of unhandled timeout perhaps? Might need to consider dropping the persistent connection completely, I'll take a look at it! How is the gateway and domoticz server connected?

[Bolten88]

Once in a while my IKEA devices stop to update. I also cannot switch any light then. When i disconnect the connection between domoticz and the IKEA hub it starts to work again. All these steps work: -Reboot domoticz -Reboot IKEA hub -Disconnect network cable from domoticz or IKEA hub Anything you have seen as well?

No, my system has been stable since the upgrade. How often does this happen?

It happens like 4 to 5 times a day. When i keep 'playing' with the lights everything is stable. When i leave it for a short while and want to check it later they do not respond anymore. I see this in the log when i try to switch a light then:

2020-09-22 11:33:46.527 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly
2020-09-22 11:34:56.539 Error: IKEA Tradfri hardware (12) thread seems to have ended unexpectedly

Some kind of unhandled timeout perhaps? Might need to consider dropping the persistent connection completely, I'll take a look at it! How is the gateway and domoticz server connected?

They are connected like this:

Router --> Switch --> Domoticz Router --> Switch --> IKEA Hub

So, domoticz and the IKEA Hub are connected to the same switch. Before the IKEA firmware update this worked without issues. I also updated to 0.9.6 now.

Oh, and for you information i allways had the polling interval on 5 seconds.

Thanks for your commitment on this!

[moroen]

Oh, and for you information i allways had the polling interval on 5 seconds.

Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

[martin-g-it]

At my side another side effect of the short polling interval (<60 seconds) tend to make the memory consumption of the Domoticz service increase a few % per day (Rpi3B+) After 10-15 days (when memory was >75%) this resulted in other functionality not responding anymore (ie. Sonos HTTP API not working anymore)

[mpranger]

My system does not respond well with pycoap, but it does respond well with coapcmd. What goes wrong with pycoap:

• Once I switched on/off/changed color a lamp, the lamp becomes unresponsive. Sometimes the response comes back, sometimes not. ==> OK with coapcmd
• With Domoticz events I scheduled a Blocky event that switches on the lights in the morning, changes the color to daylight, and then switches off again (a reset of the warm light of the evening). This morning some lights were still on, some lights were off, some lights were on with warm light. I guess this is the same problem as the unresponsiveness, but it demonstrates how unpredictability. ==> will see tomorrow if this goes OK with coapcmd

Hope this helps in getting a step further.

This sound a lot like the original problem, so I have to ask: Are you sure you've updated both pycoap and the plugin to the latest versions?

python3 plugin.py version ==> IKEA Tradfri Plugin - version 0.9.5 sudo -H pip3 install -U py3coap ==> Requirement already up-to-date: py3coap in /usr/local/lib/python3.7/dist-packages (0.9.0)

Just comitted a small patch, please try upgrading the plugin to version 0.9.6

This solved (at least) the (first) issue, thanks!

[Bolten88]

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Will try that!

[Bolten88]

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Sadly it doesn't make any difference..

[moroen]

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Sadly it doesn't make any difference..

@Bolten88 Could you please try the development branch? It isn't a fix as such, just a test to see if closing the connection as soon as possible helps with you hardware thread error...

[Bolten88]

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Sadly it doesn't make any difference..

@Bolten88 Could you please try the development branch? It isn't a fix as such, just a test to see if closing the connection as soon as possible helps with you hardware thread error...

Is this the right way to do that?

git config --global user.name Bolten88 git config --global user.email [email] git pull origin development

If yes, i did that. I see that some fields are changed at the hardware tab.

I've set 'Monitor changes' to 'Poll' and 'Observe Port' was automaticaly set to '0'.

Is that correct?

I see these errors in the log while everything seems to work properly. I will test to see if if the switches freezes again.

2020-09-23 08:45:38.033 Error: (IKEA Tradfri) 'onStop' failed 'NameError':'name 'observe_stop' is not defined'.
2020-09-23 08:45:38.033 Error: (IKEA Tradfri) ----> Line 780 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onStop
2020-09-23 08:45:38.033 Error: (IKEA Tradfri) ----> Line 623 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onStop

[Bolten88]

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Sadly it doesn't make any difference..

@Bolten88 Could you please try the development branch? It isn't a fix as such, just a test to see if closing the connection as soon as possible helps with you hardware thread error...

Is this the right way to do that?

git config --global user.name Bolten88 git config --global user.email [email] git pull origin development

If yes, i did that. I see that some fields are changed at the hardware tab.

I've set 'Monitor changes' to 'Poll' and 'Observe Port' was automaticaly set to '0'.

Is that correct?

Oh, and for you information i allways had the polling interval on 5 seconds. Thanks for your commitment on this!

Setting the polling intervall to short is know to "kill" the gateway, and the latest changes puts a bit more strain on the gateway in it self. Could you try increasing the poll-intervall (3 minutes is recommended), and see if that helps?

Sadly it doesn't make any difference..

@Bolten88 Could you please try the development branch? It isn't a fix as such, just a test to see if closing the connection as soon as possible helps with you hardware thread error...

It seems to be stable now. The only thing i see now is that the batteries are not updating with the development version.

[wvdweerd]

Thanks for the quick work-around, @moroen! :-)

It is working again for me, using the plugin 0.9.6 en py3coap 0.9.0. (also battery status is reported in Domoticz, with values that appear to be reasonable)

[moroen]

It seems to be stable now. The only thing i see now is that the batteries are not updating with the development version.

It does, but while the current master updates the domoticz device for batteries on every poll, the development version only updates the domoticz device when the value of the battery level actually has changed...

[mpranger]

Here things are not stable unfortunately. After sudo service domoticz restart things run smoothly for some hours. Then domoticz logs suddenly shows

2020-09-24 12:09:40.551 Error: IKEA Tradfri Lights hardware (4) thread seems to have ended unexpectedly

without any specific trigger. Seems to be the case with both coapcmd and pycoap...

=> update 14:14 The above happens with pycoap only. With coapcmd there is a different error:

2020-09-24 14:12:00.215 Error: (IKEA Tradfri Lights) 'onCommand' failed 'KeyError'. 2020-09-24 14:12:00.215 Error: (IKEA Tradfri Lights) ----> Line 725 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onCommand 2020-09-24 14:12:00.215 Error: (IKEA Tradfri Lights) ----> Line 645 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onCommand

[mbuurman78]

Here things are not stable unfortunately. After sudo service domoticz restart things run smoothly for some hours. Then domoticz logs suddenly shows

2020-09-24 12:09:40.551 Error: IKEA Tradfri Lights hardware (4) thread seems to have ended unexpectedly

without any specific trigger. Seems to be the case with both coapcmd and pycoap...

Same here, but also other plugins starts to be unstable since the new plugin/p3coap:

2020-09-24 13:35:05.537 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:05.537 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:05.538 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:23.613 Error: (Shelly2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883 2020-09-24 13:35:23.613 Error: (Zigbee2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883

After disabling the Ikea-plugin everything (but IKEA of course) works stable again

[Bolten88]

Here things are not stable unfortunately. After sudo service domoticz restart things run smoothly for some hours. Then domoticz logs suddenly shows 2020-09-24 12:09:40.551 Error: IKEA Tradfri Lights hardware (4) thread seems to have ended unexpectedly without any specific trigger. Seems to be the case with both coapcmd and pycoap...

Same here, but also other plugins starts to be unstable since the new plugin/p3coap:

2020-09-24 13:35:05.537 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:05.537 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:05.538 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:23.613 Error: (Shelly2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883 2020-09-24 13:35:23.613 Error: (Zigbee2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883

After disabling the Ikea-plugin everything (but IKEA of course) works stable again

I saw this as well. Other Python plugins are failing as well. Using the development version i haven't seen that error anymore.

[moroen]

Here things are not stable unfortunately. After sudo service domoticz restart things run smoothly for some hours. Then domoticz logs suddenly shows 2020-09-24 12:09:40.551 Error: IKEA Tradfri Lights hardware (4) thread seems to have ended unexpectedly without any specific trigger. Seems to be the case with both coapcmd and pycoap...

Same here, but also other plugins starts to be unstable since the new plugin/p3coap: 2020-09-24 13:35:05.537 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:05.537 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:05.538 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Shelly2MQTT hardware (17) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: Zigbee2MQTT hardware (8) thread seems to have ended unexpectedly 2020-09-24 13:35:19.541 Error: IKEA Tradfri hardware (2) thread seems to have ended unexpectedly 2020-09-24 13:35:23.613 Error: (Shelly2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883 2020-09-24 13:35:23.613 Error: (Zigbee2MQTT) Disconnected from MQTT Server: 192.168.100.150:1883 After disabling the Ikea-plugin everything (but IKEA of course) works stable again

I saw this as well. Other Python plugins are failing as well. Using the development version i haven't seen that error anymore.

Hardware threads unexpectedly ending is usually due to one plugin taking to long to process a heartbeat. So if this plugin uses to much time, it's possible to drag the other plugings down as well. What kind of hardware are you guys using? A raspberry pi is more prone to this kind of errors than a NUC. Disabling polling will probably help.

@Bolten88 If I understand you correctly, when using the development-version, you'er not seeing the IKEA Tradfri hardware thread seems to have ended unexpectedly error, and other plugins seems to be working, or? Could you please tell me what kind of hardware you're using?

[mbuurman78]

I'm using a Raspberry Pi 3b with master-version 0.9.6 and py3coap-0.9.0

[Bolten88]

sing? A raspberry pi is more prone to this kind of errors than a NUC. Disabling polling will probably help.

I'm using a raspberry pi 3b. And you are correct, since i'm using the Developer version i haven't seen the hardware errors anymore and the other (python) plugins are working as expected.

Still the IKEA plugin is not that responsive as it was. And i'm not very sure if it is without any issues yet. In the morning when i wake up i turn on 1 light bulb manualy. So after like 8 hours of no activity. And sometimes it doesn't turn on the light. Than i start some trial and error by pressing the button again, refeshing the Domoticz page and suddenly i can turn on the light witout restarting anything. So maybe i'm still a littly fuzzy then. That is why i created a script today that flashes 1 light bulb every hour. If the plugin fails for some reason i can notice that right away. But i think this a human error thing ;-)

[mpranger]

I'm using Raspberry Pi 3 Model B Plus Rev 1.3

My working containment is a cron job that restarts the domoticz service every 5 minutes

[moroen]

sing? A raspberry pi is more prone to this kind of errors than a NUC. Disabling polling will probably help.

I'm using a raspberry pi 3b. And you are correct, since i'm using the Developer version i haven't seen the hardware errors anymore and the other (python) plugins are working as expected.

Still the IKEA plugin is not that responsive as it was. And i'm not very sure if it is without any issues yet. In the morning when i wake up i turn on 1 light bulb manualy. So after like 8 hours of no activity. And sometimes it doesn't turn on the light. Than i start some trial and error by pressing the button again, refeshing the Domoticz page and suddenly i can turn on the light witout restarting anything. So maybe i'm still a littly fuzzy then. That is why i created a script today that flashes 1 light bulb every hour. If the plugin fails for some reason i can notice that right away. But i think this a human error thing ;-)

I've been running the developement version for the last few days myself, and I have observed no issues, guess it might be time to promote it to master. That the plugin is less responsive as it was, is due to the way the problems with the latest firmware has been solved, and until I get some more insigth into WHY these changes is needed, there is nothing else to do. I feel like I have had to reverse quite a few of the improvements made in previous versions.

The slower the host system, the more noticable the delay will be, the slower the network or using wifi, the longer the delay. I'm running domoticz on a old CoreDuo Mac Mini with debian, connected via ethernet cable directly to the same hub as the tradfri gateway. The delay still is noticable, but at least it appears to be stable. Sometimes it might take several seconds before a bulb turns on/off, but it always responds.

I'll keep trying to solve this another way, but until someone discovers what has changed in the latest firmware, the hopes of fixing this isn't high...

[Bolten88]

sing? A raspberry pi is more prone to this kind of errors than a NUC. Disabling polling will probably help.

I'm using a raspberry pi 3b. And you are correct, since i'm using the Developer version i haven't seen the hardware errors anymore and the other (python) plugins are working as expected. Still the IKEA plugin is not that responsive as it was. And i'm not very sure if it is without any issues yet. In the morning when i wake up i turn on 1 light bulb manualy. So after like 8 hours of no activity. And sometimes it doesn't turn on the light. Than i start some trial and error by pressing the button again, refeshing the Domoticz page and suddenly i can turn on the light witout restarting anything. So maybe i'm still a littly fuzzy then. That is why i created a script today that flashes 1 light bulb every hour. If the plugin fails for some reason i can notice that right away. But i think this a human error thing ;-)

I've been running the developement version for the last few days myself, and I have observed no issues, guess it might be time to promote it to master. That the plugin is less responsive as it was, is due to the way the problems with the latest firmware has been solved, and until I get some more insigth into WHY these changes is needed, there is nothing else to do. I feel like I have had to reverse quite a few of the improvements made in previous versions.

The slower the host system, the more noticable the delay will be, the slower the network or using wifi, the longer the delay. I'm running domoticz on a old CoreDuo Mac Mini with debian, connected via ethernet cable directly to the same hub as the tradfri gateway. The delay still is noticable, but at least it appears to be stable. Sometimes it might take several seconds before a bulb turns on/off, but it always responds.

I'll keep trying to solve this another way, but until someone discovers what has changed in the latest firmware, the hopes of fixing this isn't high...

Yeah, i'm aware of the situation why it is less responsive. I hope you are able to find a way to make it better. But if this is the way to go i can live with it. Maybe IKEA will 'fix' (reverse) the changes they made. All my automations run smoothly again since the development version. Manually clicking a switch On/Off is sometimes slow but as you say, they work!

[moroen]

Plugin version 0.9.7 has been released, no need to update py3coap or coapcmd. This version should provide better stability, but no fixes for the increased latency.

[Tinus016]

I really appreciate the amount of effort its taken. But after updating to 0.9.7 the plugin completely stops working. In other words, it can't even start.

2020-09-27 00:51:43.951 Error: (IKEA-Tradfri) Failed to initialize tradfri module.
2020-09-27 00:51:43.951 Error: (IKEA-Tradfri) Unable to find tradfricoap

This what happens when I try to switch af light on:

2020-09-27 00:51:53.971 Error: (IKEA-Tradfri) 'onHeartbeat' failed 'NameError':'name 'close_connection' is not defined'.
2020-09-27 00:51:53.972 Error: (IKEA-Tradfri) ----> Line 811 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-09-27 00:51:53.972 Error: (IKEA-Tradfri) ----> Line 768 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat

If i try to configure again it also comes whit the message:

pi@Domoticz:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config 192.168.*.* "Code"
Module 'tradfricoap' not found!

[moroen]

I really appreciate the amount of effort its taken. But after updating to 0.9.7 the plugin completely stops working. In other words, it can't even start.

2020-09-27 00:51:43.951 Error: (IKEA-Tradfri) Failed to initialize tradfri module.
2020-09-27 00:51:43.951 Error: (IKEA-Tradfri) Unable to find tradfricoap

This what happens when I try to switch af light on:

2020-09-27 00:51:53.971 Error: (IKEA-Tradfri) 'onHeartbeat' failed 'NameError':'name 'close_connection' is not defined'.
2020-09-27 00:51:53.972 Error: (IKEA-Tradfri) ----> Line 811 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-09-27 00:51:53.972 Error: (IKEA-Tradfri) ----> Line 768 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat

If i try to configure again it also comes whit the message:

pi@Domoticz:~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config 192.168.*.* "Code"
Module 'tradfricoap' not found!

Have you updated pycoap as well?

$ pip3 install -U pycoap