Shelly: Failed to create MQTT channel

[FloppyKiller]

Fehler beim hinzufügen von Shellys auf neu aufgesetzten nymea server. Installation des Servers nach dieser Anleitung: [https://nymea.io/wiki/nymea/master/install/ubuntu ]

OS Ubuntu Server 18.04LTS

Output der Debug:

I | Shelly: Found shelly thing! ZeroConfServiceEntry("shelly1-76FB03") location: "10.66.66.12":80 hostname: "shelly1-76FB03.local" domain: "local" service type: "_http._tcp" protocol: QAbstractSocket::NetworkLayerProtocol(IPv4Protocol) txt: ("id=shelly1-76FB03", "fw_id=20200320-123430/v1.6.2@514044b4", "arch=esp8266")

W | Mqtt: Unable to find a matching MQTT server port for client address "10.66.66.12" W | Shelly: Failed to create MQTT channel. W | ThingManager: Thing setup failed. Not adding thing to system.

Ports am Server sind offen:

root@nymea:~# lsof -i -P -n COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME systemd-n 74 systemd-network 16u IPv4 96983506 0t0 UDP 10.0.0.135:68 systemd-r 97 systemd-resolve 12u IPv4 96180568 0t0 UDP 127.0.0.53:53 systemd-r 97 systemd-resolve 13u IPv4 96180569 0t0 TCP 127.0.0.53:53 (LISTEN) avahi-dae 154 avahi 12u IPv4 96180397 0t0 UDP :5353 avahi-dae 154 avahi 13u IPv6 96180398 0t0 UDP :5353 avahi-dae 154 avahi 14u IPv4 96180399 0t0 UDP :47749 avahi-dae 154 avahi 15u IPv6 96180400 0t0 UDP :41444 nymead 201 root 9u IPv6 96182559 0t0 UDP :49880 nymead 201 root 12u IPv4 96181575 0t0 TCP :2222 (LISTEN) nymead 201 root 13u IPv4 96181576 0t0 TCP :4444 (LISTEN) nymead 201 root 14u IPv4 96182481 0t0 TCP :443 (LISTEN) nymead 201 root 15u IPv4 96182482 0t0 TCP :80 (LISTEN) nymead 201 root 16u IPv4 96182483 0t0 TCP :1883 (LISTEN) nymead 201 root 18u IPv4 96182572 0t0 UDP :1900 nymead 201 root 20u IPv4 96202037 0t0 TCP :1884 (LISTEN) nymead 201 root 21u IPv4 96182592 0t0 TCP 10.0.0.135:2222->10.0.11.15:55156 (ESTABLISHED) nymead 201 root 24u IPv6 96219377 0t0 TCP :2626 (LISTEN) nymead 201 root 25u IPv4 96999542 0t0 TCP 10.0.0.135:80->10.0.1.29:41238 (ESTABLISHED) nymead 201 root 26u IPv6 96999940 0t0 TCP 10.0.0.135:2626->10.0.11.15:59286 (ESTABLISHED) sshd 218 root 3u IPv4 96181299 0t0 TCP :22 (LISTEN) sshd 218 root 4u IPv6 96181313 0t0 TCP *:22 (LISTEN) master 392 root 13u IPv4 96181803 0t0 TCP 127.0.0.1:25 (LISTEN) master 392 root 14u IPv6 96181804 0t0 TCP [::1]:25 (LISTEN)

Nymea Logs nach Neustart von nymead über systemctl und erneuten Versuch einen Shelly hinzuzufügen:

root@nymea:~# journalctl -f -u nymead -- Logs begin at Tue 2020-05-05 20:13:07 UTC. -- Mai 06 07:45:07 nymea systemd[1]: nymead.service: Failed with result 'signal'. Mai 06 07:45:07 nymea systemd[1]: Stopped nymead - IoT server. Mai 06 07:45:17 nymea systemd[1]: nymead.service: Failed to reset devices.list: Operation not permitted Mai 06 07:45:17 nymea systemd[1]: Started nymead - IoT server. Mai 06 07:45:17 nymea nymead[1172]: I | Application: ===================================== Mai 06 07:45:17 nymea nymead[1172]: I | Application: nymead 0.20.0+202004051443~bionic1 started as root. Mai 06 07:45:17 nymea nymead[1172]: I | Application: ===================================== Mai 06 07:45:18 nymea nymead[1172]: W | default: QFSFileEngine::open: No file name specified Mai 06 07:45:18 nymea nymead[1172]: W | ServerManager: Could not open "" : "No file name specified" Mai 06 07:45:18 nymea nymead[1172]: W | ServerManager: Using fallback self-signed SSL certificate: "/var/lib/nymea/certs/nymead-certificate.crt" Mai 06 07:45:43 nymea nymead[1172]: W | default: Cannot find a running Bluez. Please check the Bluez installation. Mai 06 07:45:43 nymea nymead[1172]: I | Mqtt: MQTT server running on QHostAddress("0.0.0.0") : 1883 Mai 06 07:45:43 nymea nymead[1172]: I | Mqtt: MQTT server running on QHostAddress("0.0.0.0") : 1884 Mai 06 07:46:08 nymea nymead[1172]: W | Bluetooth: No bluetooth adapter found. Resource not available. Mai 06 07:46:33 nymea nymead[1172]: W | Bluez: Invalid DBus ObjectManager interface. Mai 06 07:46:33 nymea nymead[1172]: W | Nuki: Bluetooth not available Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Wemo" ( "2e3b5ce0-ecf1-43de-98f0-07df4068a583" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "OpenUv" ( "9b7d9cc8-77df-4197-a6fc-8a365747a3b1" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "TexasInstruments" ( "ae550a91-e734-4331-9d71-9f37df0b0fa6" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Netatmo" ( "69d14951-0c02-4877-bcef-dffdf48b7ccb" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Bose" ( "472a3f24-b05c-49b3-ad9a-dfda608b6760" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "NetworkDetector" ( "8e0f791e-b273-4267-8605-b7c2f55a68ab" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "UsbRelay" ( "ed0035d3-561c-498e-bdb2-2b574cbd0a2f" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "EQ3" ( "f324c43c-9680-48d8-852a-93b2227139b9" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "DateTime" ( "c26014c6-87fb-4233-85ed-01d18625018d" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "OpenWeatherMap" ( "bc6af567-2338-41d5-aac1-462dec6e4783" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "WakeOnLan" ( "b5a87848-de56-451e-84a6-edd26ad4958f" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Denon" ( "cd758269-dbbb-4ef0-80ab-48bd9a8a2765" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Nanoleaf" ( "360867ec-1594-498d-8182-fbab1fe17489" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "AirQualityIndex" ( "57d69b76-4d2d-41ec-bef6-949a79ffbe6b" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "MailNotification" ( "1ae35df1-1b51-4c93-94fa-3febc77e0318" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Tado" ( "b4f2d2ee-50bb-4786-b7f5-261fed204fa5" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "unifi" ( "88bc00c7-9ea8-4aa6-8aec-831639e8fccc" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "shelly" ( "6162773b-0435-408c-a4f8-7860d38031a9" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "tasmota" ( "d136e0c0-0cbf-4731-aabb-b2201088d6cb" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Senic" ( "413e9d77-335f-4ecf-abbc-8f2a8a399c39" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Elgato" ( "c5c03ad4-bfdb-444a-8eca-2c234c46cc27" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Kodi" ( "e7186890-99fa-4c5b-8247-09c6d450d490" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "PhilipsHue" ( "5f2e634b-b7f3-48ee-976a-b5ae22aa5c55" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "LgSmartTv" ( "4ef7a68b-9da0-4c62-b9ac-f478dc6f9f52" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "anelElektronik" ( "7a3e5b64-20e4-42bd-b86b-989b84afc22a" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "KebaKeContact" ( "9142b09f-30a9-43d0-9ede-2f8debe075ac" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "awattar" ( "9c261c33-d44e-461e-8ec1-68803cb73f12" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Pushbullet" ( "46986575-0e62-483d-b5a8-76ac356fcce7" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "Sonos" ( "cdb07719-c445-4fa5-9c7a-564ee02a4412" ) Mai 06 07:46:33 nymea nymead[1172]: W | Translations: Could not load translation "de_AT" for plugin "daylightSensor" ( "4b7d63a7-fc39-4a50-a459-457fa7653089" ) Mai 06 07:46:39 nymea nymead[1172]: I | Shelly: Found shelly thing! ZeroConfServiceEntry("shelly1-76FB03") Mai 06 07:46:39 nymea nymead[1172]: location: "10.66.66.12":80 Mai 06 07:46:39 nymea nymead[1172]: hostname: "shelly1-76FB03.local" Mai 06 07:46:39 nymea nymead[1172]: domain: "local" Mai 06 07:46:39 nymea nymead[1172]: service type: "_http._tcp" Mai 06 07:46:39 nymea nymead[1172]: protocol: QAbstractSocket::NetworkLayerProtocol(IPv4Protocol) Mai 06 07:46:39 nymea nymead[1172]: txt: ("id=shelly1-76FB03", "fw_id=20200320-123430/v1.6.2@514044b4", "arch=esp8266") Mai 06 07:46:43 nymea nymead[1172]: W | Mqtt: Unable to find a matching MQTT server port for client address "10.66.66.12" Mai 06 07:46:43 nymea nymead[1172]: W | Shelly: Failed to create MQTT channel. Mai 06 07:46:43 nymea nymead[1172]: W | ThingManager: Thing setup failed. Not adding thing to system.

locales stimmen eigentlich auch: (habe verschiedene durchprobiert: C, de_AT.UTF-8, de_DE)

root@nymea:~# locale LANG=de_AT LANGUAGE= LC_CTYPE="de_AT" LC_NUMERIC="de_AT" LC_TIME="de_AT" LC_COLLATE="de_AT" LC_MONETARY="de_AT" LC_MESSAGES="de_AT" LC_PAPER="de_AT" LC_NAME="de_AT" LC_ADDRESS="de_AT" LC_TELEPHONE="de_AT" LC_MEASUREMENT="de_AT" LC_IDENTIFICATION="de_AT" LC_ALL=

[mzanetti]

Thanks for the report. Would you mind attaching the output of ifconfig please?

Seems like the code fails to match the Shelly's IP address to a network interface on the server. Afaict from the logs the server address is 10.0.0.135 and the shelly's is 10.66.66.12.

I have created this sample code to reproduce it

    QHostAddress shelly("10.66.66.12");
    QHostAddress serverAddress("10.0.0.135");
    QHostAddress serverNetmask("255.0.0.0");
    QNetworkAddressEntry serverAddressEntry;
    serverAddressEntry.setIp(serverAddress);
    serverAddressEntry.setNetmask(serverNetmask);
    qDebug() << "Is in subnet:" << shelly.isInSubnet(serverAddressEntry.ip(), serverAddressEntry.prefixLength());

but it returns "true" so it should work. Which probably means that either I got one of the IP addresses wrong from the logs, or your subnet mask is not 255.0.0.0.

The output of ifconfig would allow me to verify both.

[FloppyKiller]

my subnet mask is /16, so 255.255.0.0 on the server network and /24 - 255.255.255.0 on the shelly network. Both networks are completely open in the firewall.

I don't have ifconfig installed, but ip a shows:

root@nymea:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 126: eth0@if127: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 56:23:bb:cc:d1:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.0.0.135/16 brd 10.0.255.255 scope global dynamic eth0 valid_lft 3352sec preferred_lft 3352sec inet6 fe80::5423:bbff:fecc:d117/64 scope link valid_lft forever preferred_lft forever

[mzanetti]

right... so as the subnetmask is /24 (which is the same as 255.255.255.0) the shelly is in a different network... With a mask of 255.255.255.0 your machine can only communicate to other hosts having IP addresses from 10.0.0.1 to 10.0.0.254.

As you're apparently in a class A network, you should set the mask to /8 (or 255.0.0.0). Then it should start working.

[mzanetti]

sorry... I've mistaken the mask a bit, but still the argument is valid...

With this ip config 10.0.0.135/16 you'd be able to communicate with hosts in 10.0.x.x, but the shelly is in 10.66.66.12/24, so it would only be able to communicate with hosts of 10.66.66.x

[mzanetti]

If you've received both ip addresses via DCHP it either hints to a bad router configuration or your administrator doesn't want you to communicate between workstations and mobile devices (which would be rather bad security tho, because the client can override the netmask anyways). Also a bit weird that the shelly got a different subnet mask than your workstation.... Would make me think now that shellies might only take the IP address from DHCP and hardcode a /24 netmask...

[FloppyKiller]

I know. But the those are two separate networks.

on net A: (server site net) root@nymea:~# ping 10.66.66.12 PING 10.66.66.12 (10.66.66.12) 56(84) bytes of data. 64 bytes from 10.66.66.12: icmp_seq=1 ttl=127 time=5.87 ms

on net B (shelly site net): SmacBook-Pro-HD:~ rcan$ ping 10.0.0.135 PING 10.0.0.135 (10.0.0.135): 56 data bytes 64 bytes from 10.0.0.135: icmp_seq=0 ttl=63 time=1.111 ms

There is one hop: SmacBook-Pro-HD:~ rcan$ traceroute 10.0.0.135 traceroute to 10.0.0.135 (10.0.0.135), 64 hops max, 52 byte packets 1 10.66.66.1 (10.66.66.1) 1.149 ms 1.141 ms 0.890 ms 2 10.0.0.135 (10.0.0.135) 1.096 ms 8.074 ms 9.905 ms

[mzanetti]

hmmm I see... so those are indeed separate networks but routed between each other... The code doesn't cope with that (yet)...

So the caveat there is that nymea needs to configure the MQTT server on the shelly... For that it needs to find out which IP address it should tell the shelly to connect to. But given both devices are in different networks, it can't know that there is a router routing between them.

I'm thinking how I could find out in the code that this would still work by just telling the shelly to use 10.0.0.135... Perhaps one option could be to add an additional parameter during setup where you'd need to specify that manually for such "complex" network setups. Or maybe I could try to resolve a route to the shelly and extract information from that... hmm... need to fiddle with this.

[FloppyKiller]

The code doesn't cope with that (yet)

Ok. I'll try adding the server into the shelly network for now.

I think this issue could be closed for now.

Thank you. best regards, Ricardo

[mzanetti]

Hmm... no... I'll keep this open... IMHO this can and should be fixed in nymea... It actually belongs to nymea itself tho, not nymea-mqtt (as the MQTT implementation itlsef really doesn't care about this stuff)

[mzanetti]

Thanks again for the report