Devices not showing in Domoticz

[mpranger]

Hi,

I just did a fresh install of the full domoticz system. (Yesterday everything was working fine, but the SD card in my Raspberry suddenly crashed due to unknown reasons, so I re-installed the full raspberry OS on a new SD card).

After installing domoticz, the IKEA tradfri plug-in, and after setting up the IKEA Tradfri plugin in the Domoticz hardware tab, the devices do not show up in the devices tab.

I have all new versions installed (I think).

I hope you can help!

python3 plugin.py version

IKEA Tradfri Plugin: 0.10.4 Tradfricoap: 0.0.17 Py3coap: 0.9.2 coapcmd: (/home/pi/domoticz/plugins/IKEA-Tradfri/bin/coapcmd) 0.1.7

The plugin.py works (list correctly shows the devices, and with on/off command I am able to turn the lights on/off)

Domoticz log shows:

_2021-01-09 21:26:41.949 Status: (IKEA Lights) Started. 2021-01-09 21:26:42.240 Status: (IKEA Lights) Entering work loop. 2021-01-09 21:26:42.241 Status: (IKEA Lights) Initialized version 0.10.4, author 'moroen' 2021-01-09 21:26:42.245 Error: (IKEA Lights) Failed to initialize tradfri module. 2021-01-09 21:26:42.245 Error: (IKEA Lights) Unable to find tradfricoap 2021-01-09 21:26:52.265 Error: (IKEA Lights) 'onHeartbeat' failed 'NameError':'name 'closeconnection' is not defined'. 2021-01-09 21:26:52.265 Error: (IKEA Lights) ----> Line 806 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat 2021-01-09 21:26:52.265 Error: (IKEA Lights) ----> Line 762 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat 2021-01-09 21:26:54.513 Status: (IKEA Lights) Started. 2021-01-09 21:26:54.760 Status: (IKEA Lights) Entering work loop. 2021-01-09 21:26:54.760 Status: (IKEA Lights) Initialized version 0.10.4, author 'moroen' 2021-01-09 21:26:54.761 Error: (IKEA Lights) Failed to initialize tradfri module. 2021-01-09 21:26:54.761 Error: (IKEA Lights) Unable to find tradfricoap 2021-01-09 21:27:00.323 (IKEA Lights) Debug logging mask set to: NONE 2021-01-09 21:27:00.304 Status: (IKEA Lights) Stop directive received. 2021-01-09 21:27:00.304 Status: (IKEA Lights) Exiting work loop. 2021-01-09 21:27:00.404 Status: (IKEA Lights) Stopping threads. 2021-01-09 21:27:00.404 Status: (IKEA Lights) Stopped.

[moroen]

@mpranger

The python framework can't find tradfricoap: 2021-01-09 21:26:42.245 Error: (IKEA Lights) Unable to find tradfricoap

In my opinion, one of the biggest headaches with regards to python, is that it's quite possible to end up in a situation where the python3 environment run from the command line is not the same as the one when using embedded python3.

Take a look at: https://github.com/moroen/IKEA-Tradfri-plugin/wiki/Common-errors

[bengt-a]

Hi @moroen, I have to first say that your plugin is AWESOME! It is amazing to be able to control my new IKEA kitchen lights from Domoticz.

However, today I added a new remote and some new devices, but they are not showing up in Domoticz. Perhaps I am just doing something wrong, but I thought they would appear automagically - perhaps after restarting domoticz?

python3 plugin.py version
IKEA Tradfri Plugin: 0.10.4
Py3coap: 0.9.2
coapcmd: Not found (Looking for /home/pi/domoticz/plugins/IKEA-Tradfri/bin/coapcmd).

Tradfricoap: 0.0.17

The old devices are showing up and are fine. But the new ones don't show up...

I tried installing py3coap and got

sudo pip3 install py3coap
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: py3coap in /usr/local/lib/python3.5/dist-packages (0.9.2)
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config api py3coap

I feel a bit confused since coapmd seems to be working, but py3coap doesn't? Or is it the other way around?

pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config api coapcmd
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config gw 192.168.1.121 4KPxxxxxxxxxxxxx
Traceback (most recent call last):
  File "plugin.py", line 133, in <module>
    process_args(args)
  File "/usr/local/lib/python3.5/dist-packages/tradfricoap/cli.py", line 102, in process_args
    set_config(args)
  File "/usr/local/lib/python3.5/dist-packages/tradfricoap/cli.py", line 214, in set_config
    create_ident(args.IP, args.KEY, conf_object)
  File "/usr/local/lib/python3.5/dist-packages/tradfricoap/coapcmd_api.py", line 165, in create_ident
    stdout=subprocess.PIPE,
  File "/usr/lib/python3.5/subprocess.py", line 383, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/home/pi/domoticz/plugins/IKEA-Tradfri/bin/coapcmd'
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config api py3coap
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ python3 plugin.py config gw 192.168.1.121 4KPxxxxxxxxxxxxx
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $

[moroen]

@bengt-a

py3coap is working, coapcmd is not... To use coapcmd, you need to copy the coapcmd command for your arch to the bin directory in the IKEA-Tradfri folder.

Try listing the devices from the command line:

$ python3 plugin.py list

If the devices appears in the list, they should appear after a domoticz restart (or selecting the plugin on the hardware page and press the "Update"-button)

[bengt-a]

@moroen Thanks. When I run the install_coapcmd.sh script, it doesn't seem to install anything. There is no bin directory, and the script doesn't exit with any exit code.

I see that it should finish with

if [ $? -eq 0 ]
then
  echo "coapcmd installed as bin/coapcmd"
else
  echo "Installation of coapcmd failed" >&2
fi

But when I execute the install script I only get:

pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $ sh install_coapcmd.sh
Checking git... Ok
Checking go... Ok
github.com/moroen/coapcmd (download)
github.com/mitchellh/go-homedir (download)
github.com/moroen/gocoap (download)
github.com/dustin/go-coap (download)
github.com/moroen/dtls (download)
github.com/spf13/cobra (download)
github.com/spf13/pflag (download)
github.com/spf13/viper (download)
github.com/fsnotify/fsnotify (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
Fetching https://golang.org/x/sys/internal/unsafeheader?go-get=1
Parsing meta tags from https://golang.org/x/sys/internal/unsafeheader?go-get=1 (status code 200)
get "golang.org/x/sys/internal/unsafeheader": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/internal/unsafeheader?go-get=1
get "golang.org/x/sys/internal/unsafeheader": verifying non-authoritative meta tag
package math/bits: unrecognized import path "math/bits" (import path does not begin with hostname)
github.com/hashicorp/hcl (download)
github.com/magiconair/properties (download)
github.com/mitchellh/mapstructure (download)
github.com/pelletier/go-toml (download)
github.com/spf13/afero (download)
Fetching https://golang.org/x/text/transform?go-get=1
Parsing meta tags from https://golang.org/x/text/transform?go-get=1 (status code 200)
get "golang.org/x/text/transform": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/transform?go-get=1
get "golang.org/x/text/transform": verifying non-authoritative meta tag
Fetching https://golang.org/x/text?go-get=1
Parsing meta tags from https://golang.org/x/text?go-get=1 (status code 200)
golang.org/x/text (download)
Fetching https://golang.org/x/text/unicode/norm?go-get=1
Parsing meta tags from https://golang.org/x/text/unicode/norm?go-get=1 (status code 200)
get "golang.org/x/text/unicode/norm": found meta tag main.metaImport{Prefix:"golang.org/x/text", VCS:"git", RepoRoot:"https://go.googlesource.com/text"} at https://golang.org/x/text/unicode/norm?go-get=1
get "golang.org/x/text/unicode/norm": verifying non-authoritative meta tag
github.com/spf13/cast (download)
github.com/spf13/jwalterweatherman (download)
github.com/subosito/gotenv (download)
Fetching https://gopkg.in/ini.v1?go-get=1
Parsing meta tags from https://gopkg.in/ini.v1?go-get=1 (status code 200)
get "gopkg.in/ini.v1": found meta tag main.metaImport{Prefix:"gopkg.in/ini.v1", VCS:"git", RepoRoot:"https://gopkg.in/ini.v1"} at https://gopkg.in/ini.v1?go-get=1
gopkg.in/ini.v1 (download)
Fetching https://gopkg.in/yaml.v2?go-get=1
Parsing meta tags from https://gopkg.in/yaml.v2?go-get=1 (status code 200)
get "gopkg.in/yaml.v2": found meta tag main.metaImport{Prefix:"gopkg.in/yaml.v2", VCS:"git", RepoRoot:"https://gopkg.in/yaml.v2"} at https://gopkg.in/yaml.v2?go-get=1
gopkg.in/yaml.v2 (download)
pi@domo2 ~/domoticz/plugins/IKEA-Tradfri $

So I am not sure where to find the coapcmd command so I can put it in the bin directory. Actually, there isn't even a bin directory. (I also tried manually creating the bin directory and re-running the install script, but that didn't change anything...)

UPDATE: The devices appear in the list command, and apparently also added themselves to Domoticz overnight. :) So that part is now solved. But I still probably want get coapcmd to work, right?

Thanks for your assistance!

[mpranger]

@mpranger

The python framework can't find tradfricoap: 2021-01-09 21:26:42.245 Error: (IKEA Lights) Unable to find tradfricoap

In my opinion, one of the biggest headaches with regards to python, is that it's quite possible to end up in a situation where the python3 environment run from the command line is not the same as the one when using embedded python3.

Take a look at: https://github.com/moroen/IKEA-Tradfri-plugin/wiki/Common-errors

Thanks for the quick reply! I'm a bit of a noob, so I don't fully understand what to do here. And because of that, I run into issues. If I follow the guide and do:

$ cd plugins/IKEA-Tradfri $ git clone https://github.com/moroen/tradfricoap.git

but when I then do:

python3 plugin.py list

I get

Tradfricoap appears to be installed both as a system- and local module. Please remove the tradfricoap directory before running the plugin!

When I remove the directory /domoticz/plugins/IKEA-Tradfri, the problem remains the same obviously. I guess I am removing the wrong directory (either system or local), but I don't know where the other directory is located. Can you give a bit more step-by-step guidance? Thanks again for your assistance!

[moroen]

@bengt-a

UPDATE: The devices appear in the list command, and apparently also added themselves to Domoticz overnight. :) So that part is now solved. But I still probably want get coapcmd to work, right?

You don't need coapcmd if py3coap is working. Py3coap is the default, recommended option, but coapcmd exists as a backup/alternative on systems where it's not possible to get py3coap installed, such as some NAS-systems (and on Windows).

So if you see and can switch devices from domoticz, you're all set!

[moroen]

@mpranger

Try uninstalling the system py3coap module:

$ sudo pip3 uninstall py3coap

and then try to clone tradfricoap into the plugin directory.

[bengt-a]

@moroen

You don't need coapcmd if py3coap is working. Py3coap is the default, recommended option, but coapcmd exists as a backup/alternative on systems where it's not possible to get py3coap installed, such as some NAS-systems (and on Windows).

So if you see and can switch devices from domoticz, you're all set!

Thank you for the most useful plugin ever! Well done!

[mpranger]

sudo pip3 uninstall py3coap

this gives

Skipping py3coap as it is not installed.

while

python3 plugin.py version

still gives

IKEA Tradfri Plugin: 0.10.4 Tradfricoap: 0.0.17 Py3coap: 0.9.2 coapcmd: (/home/pi/domoticz/plugins/IKEA-Tradfri/bin/coapcmd) 0.1.7

Sorry for the inconvenience, but I have no clue what is going wrong here...

[mpranger]

sudo pip3 uninstall py3coap

this gives

Skipping py3coap as it is not installed.

while

python3 plugin.py version

still gives

IKEA Tradfri Plugin: 0.10.4 Tradfricoap: 0.0.17 Py3coap: 0.9.2 coapcmd: (/home/pi/domoticz/plugins/IKEA-Tradfri/bin/coapcmd) 0.1.7

Sorry for the inconvenience, but I have no clue what is going wrong here...

Switching to coapcmd suddenly helped. Now the lights are visible in domoticz as devices, and on/off works. Although it now works, I don't like that I don't understand it :-) Perhaps you can share your thoughts?

BTW Fully agree with @bengt-a , the plugin is awesome!

[moroen]

@mpranger

I think your problem might be inconsistent use of sudo...

Try pip3 uninstall without sudo first, and then again with sudo:

$ pip3 uninstall tradfricoap py3coap
$ sudo pip3 uninstall tradfricoap py3coap

Repeat until you get "Skipping ... as it is not installed." both with and without sudo. Then python3 plugin.py version should correctly show that py3coap and tradfricoap isn't installed.

Log in (or su -) as the user that's being used to run domoticz (this is the pi user if you're on a Raspberry, and should never, ever be the root user), and use the --user option to make sure that the modules are installed in a place where domoticz (when run as that user) should be able to find them:

$ pip3 install --user py3coap tradfricoap

Then, as the same user, try python3 plugin.py version, and python3 plugin.py list, and if that works, still as the same user, start domoticz from the command line. If that works, just make sure that systemd is actually running domoticz as the correct user.

And @bengt-a and @mpranger, thank you for your kind words!

[mpranger]

Thanks for the help @moroen It now works correctly.

I did

$ sudo pip3 uninstall tradfricoap py3coap
$ pip3 uninstall tradfricoap py3coap

until WARNING: Skipping py3coap/tradfricoap as it is not installed.

Then did

$ pip3 install --user py3coap tradfricoap

but this still resulted in the same issues in domoticz. Uninstalled py3coap/tradfricoap according to the same process again. Then did:

$ cd /home/pi/domoticz/plugins/IKEA-Tradfri/
$ git clone https://github.com/moroen/tradfricoap.git
$ pip3 install py3coap
$ sudo service domoticz restart

and both coapcmd and py3coap are working fine, also in domoticz.

Let me know if you have any questions, otherwise the issue can be closed.

[moroen]

The way Python handles it's environment is a powerful feature, but also a potential quagmire of conflicts, modules not found, several versions of modules (and several versions of python) and a host of other problems... In my opinion it is both the most powerful and most annoying feature of a language that in every other respect is very enjoyable to work with...

Why py3coap works, but you still need to clone tradfricoap manually is baffling, in all rights, py3coap and tradfricoap both should be installed in the same location when using the --user option... but glad you got it working!