jens-maus / RaspberryMatic

:house: A feature-rich but lightweight, buildroot-based Linux operating system alternative for your CloudFree CCU3/ELV-Charly 'homematicIP CCU' IoT smarthome central. Running as a pure virtual appliance (ProxmoxVE, Home Assistant, LXC, Docker/OCI, Kubernetes/K8s, etc.) or on a dedicated embedded device (RaspberryPi, Tinkerboard, IntelNUC, etc.)
https://raspberrymatic.de
Apache License 2.0
1.5k stars 184 forks source link

DNS resolution within raspberymatic docker container fails #2739

Open cubed-it opened 1 month ago

cubed-it commented 1 month ago

Describe the issue you are experiencing

Hello everyone,

I have set up a raspberry latest distro with docker + raspberymatic and have problems with DNS resolution within the container.

I have created the container exactly according to the following blueprint: https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#using-docker-compose

services:
  homematic:
    image: ghcr.io/jens-maus/raspberrymatic:latest
    container_name: homematic
    hostname: homematic
    read_only: true
    privileged: true
    restart: unless-stopped
    stop_grace_period: 30s
    volumes:
      - /home/<user>/docker/homematic:/usr/local:rw
      - /lib/modules:/lib/modules:ro
      - /run/udev/control:/run/udev/control
    networks:
      homematic:
        ipv4_address: 192.168.145.9  

networks:
  homematic:
    name: homematic
    driver: macvlan
    driver_opts:
      parent: br0
    ipam:
      config:
        - subnet: 192.168.145.0/24
          gateway: 192.168.145.1

The issue is now, that no DNS resolution is possible from within the container. journalctl gives me the following messages cyclically:

May 11 13:32:42 rpi-le dockerd[674]: time="2024-05-11T13:32:42.524563592+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:192.168.145.9:37355" dns-server="udp:192.168.145.1:53" error="read udp 192.168.145.9:37355->192.168.145.1:53: i/o timeout" question=";google.com.fritz.box.\tIN\t A" spanID=5a987ed8189bc5af traceID=4df1d4d99890b675013fa3b27e2218d9

A CUxD script issuing a query fails and I have a alert inside raspberymatic web interface: WatchDog: no-internet | ausgelöst | No internet connection detected

Through an nslookup wdr.de from inside the container:

/ # nslookup wdr.de
Server:         127.0.0.11
Address:        127.0.0.11:53

;; connection timed out; no servers could be reached

and journalctl:

May 11 13:37:19 rpi-le dockerd[674]: time="2024-05-11T13:37:19.571985054+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:192.168.145.9:43799" dns-server="udp:192.168.145.1:53" error="read udp 192.168.145.9:43799->192.168.145.1:53: i/o timeout" question=";wdr.de.\tIN\t A" spanID=9d703cbb25db02d6 traceID=653abed0a8d117b93f0e74009d1e4ea9

However, if I completely reset the iptables within the container:

iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -t raw -F
iptables -t raw -X
iptables -t security -F
iptables -t security -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

The nslookup wdr.de is now working from inside the container:

/ # nslookup wdr.de
Server:         127.0.0.11
Address:        127.0.0.11:53

Non-authoritative answer:

Non-authoritative answer:
Name:   wdr.de
Address: 149.219.209.51

Also my CUxD script is now working.

I have reached the end of my knowledge, what am I doing wrong or how can I solve the iptables problem properly and persistent? Thanks!

Ps: I have now also played around a bit with the firewall settings within the web interface. Adding port 53 does not help, only the option ports open is so far successful. I would like to see a proper solution, but in principle that would be fine with me. Is ports open a security risk in a private LAN?

Pss: Resolution is working after adding try_exec_cmd "/usr/sbin/iptables -A INPUT -p udp --source-port 53 -j ACCEPT" to libfirewall.tcl and mapping it inside the container. That solution is however not very good, since I have to replace the complete file.... Adding the port via web interface results in:

ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain state NEW
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain

But what seems to be required is just this one rule:

ACCEPT     udp  --  anywhere             anywhere             udp spt:domain

Describe the behavior you expected

DNS requests should resolve

Steps to reproduce the issue

  1. Run rapsberrymatic as described https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI#using-docker-compose
  2. exec into the container
  3. nslookup anything

What is the version this bug report is based on?

3.75.7.20240420

Which base platform are you running?

rpi2 (RaspberryPi2, ARM/armhf)

Which HomeMatic/homematicIP radio module are you using?

n/a

Anything in the logs that might be useful for us?

May 11 13:32:07 homematic syslog.info syslogd started: BusyBox v1.36.1
May 11 13:32:10 homematic user.info firewall: configuration set
May 11 13:32:44 homematic daemon.err xinetd[1044]: Unable to read included directory: /etc/config/xinetd.d [file=/etc/xinetd.conf] [line=14]
May 11 13:32:44 homematic daemon.crit xinetd[1044]: 1044 {init_services} no services. Exiting...
May 11 13:32:45 homematic daemon.info cuxd[1091]: CUx-Daemon(2.11) on CCU(3.75.7.20240420) start PID:1091
May 11 13:32:45 homematic daemon.info cuxd[1091]: write_pid /var/run/cuxd.pid [1091]
May 11 13:32:45 homematic daemon.info cuxd[1091]: load paramsets(/usr/local/addons/cuxd/cuxd.ps) size:15 update(-62s):Sat May 11 13:31:43 2024
May 11 13:32:45 homematic daemon.info cuxd[1091]: 0 device-paramset(s) loaded ok!
May 11 13:32:45 homematic daemon.info cuxd[1091]: write_proxy /var/cache/cuxd_proxy.ini (1091 /usr/local/addons/cuxd/ 2.11 3.75.7.20240420 0)
May 11 13:32:45 homematic daemon.info cuxd[1091]: add interface 'CUxD'
May 11 13:32:45 homematic user.info cuxd: started cux-daemon
May 11 13:32:45 homematic daemon.info cuxd[1091]: write interface(1) 'VirtualDevices' to /usr/local/etc/config/InterfacesList.xml
May 11 13:32:45 homematic daemon.info cuxd[1091]: write interface(2) 'CUxD' to /usr/local/etc/config/InterfacesList.xml
May 11 13:33:01 homematic daemon.warn cuxd[1091]: process_rpc_request(127.0.0.1) - illegal XMLRPC(listDevices) request
May 11 13:33:08 homematic daemon.warn cuxd[1091]: process_rpc_request(127.0.0.1) - illegal XMLRPC(init) request
May 11 13:33:12 homematic daemon.info : starting pid 1277, tty '': '/bin/mv /tmp/boot.log /var/log/boot.log'
May 11 13:33:12 homematic daemon.info : starting pid 1278, tty '/dev/null': '/usr/bin/monit -Ic /etc/monitrc'
May 11 13:33:12 homematic user.info monit[1278]: Starting Monit 5.33.0 daemon with http interface at /var/run/monit.sock
May 11 13:33:12 homematic user.info monit[1278]: 'homematic' Monit 5.33.0 started
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: Error 1 at row 11 col 36 near ^(";", 1); } }  Write(upl);^M  [SyntaxError():iseESP.cpp:1149]
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: (";", 1); } }  Write(upl); [ParseProgram():iseESP.cpp:386]
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: Error 1 at row 11 col 36 near ^(";", 1); } }  Write(upl);^M  [SyntaxError():iseESP.cpp:1149]
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: (";", 1); } }  Write(upl); [ParseProgram():iseESP.cpp:386]
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: Error 1 at row 11 col 36 near ^(";", 1); } }  Write(upl);^M  [SyntaxError():iseESP.cpp:1149]
May 11 13:33:12 homematic local0.err ReGaHss: ERROR: SyntaxError: (";", 1); } }  Write(upl); [ParseProgram():iseESP.cpp:386]
May 11 13:33:27 homematic user.err monit[1278]: 'sshdEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'hs485dEnabled' status failed (2) -- grep: /var/etc/hs485d.conf: No such file or directory
May 11 13:33:27 homematic user.err monit[1278]: 'multimacdEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'hmlangwEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'rfdEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.warn monit[1278]: 'hasUSB' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'hb_rf_eth-CheckEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'coProcessorCheck' status failed (1) -- no output
May 11 13:33:27 homematic user.warn monit[1278]: 'rpi4usb3Check' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: 'tailscaleEnabled' status failed (1) -- no output
May 11 13:33:27 homematic user.err monit[1278]: Lookup for '/media/usb1' filesystem failed  -- not found in /proc/self/mounts
May 11 13:33:27 homematic user.err monit[1278]: Filesystem '/media/usb1' not mounted
May 11 13:33:27 homematic user.err monit[1278]: 'usb1' unable to read filesystem '/media/usb1' state
May 11 13:33:27 homematic user.info monit[1278]: 'usb1' trying to restart
May 11 13:33:28 homematic daemon.info cuxd[1091]: INIT 'xmlrpc_bin://127.0.0.1:31999' '1040'
May 11 13:33:28 homematic daemon.info cuxd[1091]: RPC-server from HM-CCU (1040) registered!
May 11 13:33:29 homematic daemon.info cuxd[1091]: connection to 127.0.0.1:8183 successfull!
May 11 13:33:43 homematic user.warn monit[1278]: 'hasUSB' status failed (1) -- no output
May 11 13:33:43 homematic user.warn monit[1278]: 'rpi4usb3Check' status failed (1) -- no output
May 11 13:33:43 homematic user.err monit[1278]: Filesystem '/media/usb1' not mounted
May 11 13:33:43 homematic user.err monit[1278]: 'usb1' unable to read filesystem '/media/usb1' state
May 11 13:33:43 homematic user.info monit[1278]: 'usb1' trying to restart
May 11 13:33:43 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:33:59 homematic user.warn monit[1278]: 'hasUSB' status failed (1) -- no output
May 11 13:33:59 homematic user.err monit[1278]: 'rpi4usb3Check' status failed (1) -- no output
May 11 13:33:59 homematic user.err monit[1278]: Filesystem '/media/usb1' not mounted
May 11 13:33:59 homematic user.err monit[1278]: 'usb1' unable to read filesystem '/media/usb1' state
May 11 13:33:59 homematic user.info monit[1278]: 'usb1' trying to restart
May 11 13:33:59 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:34:15 homematic user.warn monit[1278]: 'hasUSB' status failed (1) -- no output
May 11 13:34:15 homematic user.err monit[1278]: Filesystem '/media/usb1' not mounted
May 11 13:34:15 homematic user.err monit[1278]: 'usb1' unable to read filesystem '/media/usb1' state
May 11 13:34:15 homematic user.info monit[1278]: 'usb1' trying to restart
May 11 13:34:15 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:34:31 homematic user.err monit[1278]: 'hasUSB' status failed (1) -- no output
May 11 13:34:31 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:34:47 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:35:03 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:35:19 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:35:34 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:35:50 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:36:06 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:36:22 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:36:38 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:36:54 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:37:09 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:37:25 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:37:41 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:37:57 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:38:13 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:38:29 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:38:44 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:39:00 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:39:16 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:39:32 homematic user.warn monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:39:48 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:39:48 homematic user.info monit[1278]: 'internetCheck' exec: '/bin/triggerAlarm.tcl No internet connection detected WatchDog: no-internet true'
May 11 13:40:04 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:40:20 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:40:35 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:40:51 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:41:07 homematic user.err monit[1278]: 'internetCheck' status failed (1) -- no output
May 11 13:41:23 homematic user.info monit[1278]: 'internetCheck' status succeeded (0) -- no output

Additional information

For completeness, here is my network configuration:

nmcli con mod 'Wired connection 1' con-name eth0
nmcli con add ifname br0 type bridge con-name br0
nmcli con add type bridge-slave ifname eth0 master br0
nmcli con mod br0 bridge.stp no

nmcli con mod br0 ipv4.addresses 192.168.145.5/24
nmcli con mod br0 ipv4.gateway 192.168.145.1
nmcli con mod br0 ipv4.dns '192.168.145.1'
nmcli con mod br0 ipv4.dns-search 'fritz.box'
nmcli con mod br0 ipv4.method manual

nmcli con down eth0 && nmcli con up br0
systemctl restart NetworkManager.service

nmcli device show

GENERAL.DEVICE:                         br0
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         92:C9:C1:05:3E:F7
GENERAL.MTU:                            1400
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     br0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/5
IP4.ADDRESS[1]:                         192.168.145.5/24
IP4.GATEWAY:                            192.168.145.1
IP4.ROUTE[1]:                           dst = 0.0.0.0/0, nh = 192.168.145.1, mt = 425
IP4.ROUTE[2]:                           dst = 192.168.145.0/24, nh = 0.0.0.0, mt = 425
IP4.DNS[1]:                             192.168.145.1
IP4.SEARCHES[1]:                        fritz.box
IP6.ADDRESS[1]:                         fe80::d186:af11:7f56:3c04/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 1024

GENERAL.DEVICE:                         br-5dc32aee5a36
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         02:42:08:5C:1F:AD
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     br-5dc32aee5a36
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/2
IP4.ADDRESS[1]:                         172.18.0.1/16
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 172.18.0.0/16, nh = 0.0.0.0, mt = 0
IP6.ADDRESS[1]:                         fe80::42:8ff:fe5c:1fad/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256

GENERAL.DEVICE:                         br-95ef8c538bf6
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         02:42:3F:4C:40:6B
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     br-95ef8c538bf6
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/3
IP4.ADDRESS[1]:                         172.21.0.1/16
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 172.21.0.0/16, nh = 0.0.0.0, mt = 0
IP6.ADDRESS[1]:                         fe80::42:3fff:fe4c:406b/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256

GENERAL.DEVICE:                         lo
GENERAL.TYPE:                           loopback
GENERAL.HWADDR:                         00:00:00:00:00:00
GENERAL.MTU:                            65536
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     lo
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/1
IP4.ADDRESS[1]:                         127.0.0.1/8
IP4.GATEWAY:                            --
IP6.ADDRESS[1]:                         ::1/128
IP6.GATEWAY:                            --

GENERAL.DEVICE:                         br-d81aeccab029
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         02:42:6F:BB:F1:9E
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     br-d81aeccab029
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/4
IP4.ADDRESS[1]:                         172.23.0.1/16
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 172.23.0.0/16, nh = 0.0.0.0, mt = 0
IP6.GATEWAY:                            --

GENERAL.DEVICE:                         docker0
GENERAL.TYPE:                           bridge
GENERAL.HWADDR:                         02:42:51:70:C1:12
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     docker0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/6
IP4.ADDRESS[1]:                         172.17.0.1/16
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 172.17.0.0/16, nh = 0.0.0.0, mt = 0
IP6.GATEWAY:                            --

GENERAL.DEVICE:                         tap0
GENERAL.TYPE:                           tun
GENERAL.HWADDR:                         92:C9:C1:05:3E:F7
GENERAL.MTU:                            1400
GENERAL.STATE:                          100 (connected (externally))
GENERAL.CONNECTION:                     tap0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/8
IP4.GATEWAY:                            --
IP6.ADDRESS[1]:                         fe80::90c9:c1ff:fe05:3ef7/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256

GENERAL.DEVICE:                         eth0
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         B8:27:EB:7A:8F:E7
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     bridge-slave-eth0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/7
WIRED-PROPERTIES.CARRIER:               on
IP4.GATEWAY:                            --
IP6.GATEWAY:                            --

GENERAL.DEVICE:                         veth957ff27
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         52:2A:88:32:62:B8
GENERAL.MTU:                            1500
GENERAL.STATE:                          10 (unmanaged)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
WIRED-PROPERTIES.CARRIER:               on
IP4.GATEWAY:                            --
IP6.ADDRESS[1]:                         fe80::502a:88ff:fe32:62b8/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256

GENERAL.DEVICE:                         vethe607849
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         1A:6E:D0:2D:AC:8C
GENERAL.MTU:                            1500
GENERAL.STATE:                          10 (unmanaged)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
WIRED-PROPERTIES.CARRIER:               on
IP4.GATEWAY:                            --
IP6.ADDRESS[1]:                         fe80::186e:d0ff:fe2d:ac8c/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256