christgau / wsdd

A Web Service Discovery host daemon.
MIT License
808 stars 97 forks source link

FreeBSD 31.1 running wsdd in Jail OSError: [Errno 48] Address already in use #160

Closed jemail99 closed 1 year ago

jemail99 commented 1 year ago

OS: FreeBSD 13.1 on Raspberry Pi4 8GB wsdd: latest version installed by pkg

Running wsdd in a freeBSD jail produces the following error:

wsdd -v -i genet0 -w HOME
2022-12-06 12:15:15,565:wsdd INFO(pid 62058): using pre-defined UUID 2cc44299-b932-5678-8db7-c5b69e20df08
Traceback (most recent call last):
File "/usr/local/bin/wsdd", line 1799, in <module>
sys.exit(main())
File "/usr/local/bin/wsdd", line 1754, in main
nm = RouteSocketAddressMonitor(aio_loop)
File "/usr/local/bin/wsdd", line 1058, in __call__
obj.enumerate()
File "/usr/local/bin/wsdd", line 1097, in enumerate
self.do_enumerate()
File "/usr/local/bin/wsdd", line 1471, in do_enumerate
self.parse_route_socket_response(rt_buf.raw, True)
File "/usr/local/bin/wsdd", line 1505, in parse_route_socket_response
new_intf = self.parse_addrs(buf, sa_offset, offset + rtm_len, intf, addr_mask, rtm_type, intf_flags)
File "/usr/local/bin/wsdd", line 1551, in parse_addrs
self.handle_new_address(addr, addr_family, intf)
File "/usr/local/bin/wsdd", line 1161, in handle_new_address
mch = MulticastHandler(addr_family, addr, interface, self.aio_loop)
File "/usr/local/bin/wsdd", line 83, in __init__
self.init_v4()
File "/usr/local/bin/wsdd", line 163, in init_v4
self.uc_send_socket.bind((self.address, WSD_UDP_PORT))
OSError: [Errno 48] Address already in use

In the jail:

# sockstat -4 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
root     smbd       95235 47 tcp4   10.7.4.101:445        *:*
root     smbd       95235 48 tcp4   10.7.4.101:139        *:*
root     rsync      53218 5  tcp4   10.7.4.101:873        *:*
syncthing syncthing 44257 33 udp4   10.7.4.101:36995      *:*
syncthing syncthing 44257 34 udp4   10.7.4.101:21027      *:*
syncthing syncthing 44257 35 tcp4   10.7.4.101:54225      *:*
syncthing syncthing 44257 38 tcp4   10.7.4.101:8384       *:*
radicale python3.9  34094 5  tcp4   10.7.4.101:54452      *:*
root     qbittorren 31295 12 tcp4   10.7.4.101:53000      *:*
root     qbittorren 31295 13 udp4   10.7.4.101:53000      *:*
root     qbittorren 31295 23 tcp4   10.7.4.101:53888      *:*

# sockstat -6 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS

On the host:

# sockstat -4 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
root     smbd       95235 47 tcp4   10.7.4.101:445        *:*
root     smbd       95235 48 tcp4   10.7.4.101:139        *:*
dlna     java       24749 81 tcp4   10.7.4.104:23423      *:*
dlna     java       24749 102 tcp4  10.7.4.104:23523      *:*
dlna     java       24749 112 tcp4  10.7.4.104:23424      *:*
dlna     java       24749 122 tcp4  10.7.4.104:23524      *:*
dlna     java       24749 146 tcp4  10.7.4.104:8895       *:*
dlna     java       24749 147 udp4  10.7.4.104:1900       *:*
dlna     java       24749 148 udp4  10.7.4.104:48302      *:*
root     rsync      53218 5  tcp4   10.7.4.101:873        *:*
syncthing syncthing 44257 33 udp4   10.7.4.101:36995      *:*
syncthing syncthing 44257 34 udp4   10.7.4.101:21027      *:*
syncthing syncthing 44257 35 tcp4   10.7.4.101:54225      *:*
syncthing syncthing 44257 38 tcp4   10.7.4.101:8384       *:*
radicale python3.9  34094 5  tcp4   10.7.4.101:54452      *:*
root     qbittorren 31295 12 tcp4   10.7.4.101:53000      *:*
root     qbittorren 31295 13 udp4   10.7.4.101:53000      *:*
root     qbittorren 31295 23 tcp4   10.7.4.101:53888      *:*
root     sshd       53876 3  tcp4   10.7.4.100:53869      *:*
ntpd     ntpd       74352 21 udp4   *:123                 *:*
ntpd     ntpd       74352 22 udp4   10.7.4.100:123        *:*
ntpd     ntpd       74352 25 udp4   127.0.0.1:123         *:*
ntpd     ntpd       74352 27 udp4   10.7.4.101:123        *:*
ntpd     ntpd       74352 28 udp4   10.7.4.104:123        *:*

# sockstat -6 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
ntpd     ntpd       74352 20 udp6   *:123                 *:*
ntpd     ntpd       74352 23 udp6   ::1:123               *:*
ntpd     ntpd       74352 24 udp6   fe80::1%lo0:123       *:*

Any help appreciated.

christgau commented 1 year ago

Could you please run wsdd with an additional -v, like wsdd -vv -i genet0 -w HOME. Additionally, what's the output of ifconfig genet0 ("hide" prefixes of public addresses, in case of concerns)?

jemail99 commented 1 year ago

Running wsdd with -vv option:

#wsdd -vv -i genet0 -w HO
2022-12-07 08:32:32,539:wsdd INFO(pid 56041): using pre-defined UUID 2cc44299-b932-5678-8db7-c5b69e20df08
2022-12-07 08:32:32,541:asyncio DEBUG(pid 56041): Using selector: KqueueSelector
2022-12-07 08:32:32,575:wsdd DEBUG(pid 56041): new address 10.7.4.101 on genet0
2022-12-07 08:32:32,577:wsdd DEBUG(pid 56041): handling traffic for 10.7.4.101 on genet0
Traceback (most recent call last):
File "/usr/local/bin/wsdd", line 1799, in <module>
sys.exit(main())
File "/usr/local/bin/wsdd", line 1754, in main
nm = RouteSocketAddressMonitor(aio_loop)
File "/usr/local/bin/wsdd", line 1058, in __call__
obj.enumerate()
File "/usr/local/bin/wsdd", line 1097, in enumerate
self.do_enumerate()
File "/usr/local/bin/wsdd", line 1471, in do_enumerate
self.parse_route_socket_response(rt_buf.raw, True)
File "/usr/local/bin/wsdd", line 1505, in parse_route_socket_response
new_intf = self.parse_addrs(buf, sa_offset, offset + rtm_len, intf, addr_mask, rtm_type, intf_flags)
File "/usr/local/bin/wsdd", line 1551, in parse_addrs
self.handle_new_address(addr, addr_family, intf)
File "/usr/local/bin/wsdd", line 1161, in handle_new_address
mch = MulticastHandler(addr_family, addr, interface, self.aio_loop)
File "/usr/local/bin/wsdd", line 83, in __init__
self.init_v4()
File "/usr/local/bin/wsdd", line 163, in init_v4
self.uc_send_socket.bind((self.address, WSD_UDP_PORT))
OSError: [Errno 48] Address already in use

ifconfig on host:

# ifconfig genet0
genet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether xx:xx:xx:xx:xx:xx
inet 10.7.4.100 netmask 0xffffff00 broadcast 10.7.4.255
inet 10.7.4.101 netmask 0xffffffff broadcast 10.7.4.101
inet 10.7.4.104 netmask 0xffffffff broadcast 10.7.4.104
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ifconfig in jail:

# ifconfig genet0
genet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether xx:xx:xx:xx:xx:xx
inet 10.7.4.101 netmask 0xffffffff broadcast 10.7.4.101
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
christgau commented 1 year ago

Mh, I'm still confused. I've found a similar issue over here, but no solution. Can you please post your jail config plus (for completeness) the jail-related sysctls (security.jail.*) which hopefully enables me to reproduce the issue. And: Have you run wsdd successfully outside the jail?

jemail99 commented 1 year ago

Host: sysctl.conf

security.jail.mount_allowed=1
security.jail.mount_nullfs_allowed=1
security.jail.param.allow.mount.nullfs=1
security.jail.mount_devfs_allowed=1
security.jail.param.allow.mount.devfs=1
security.jail.mount_zfs_allowed=1
security.jail.param.allow.mount.zfs=1

sysctl on command line:

sysctl -a | grep security.jail
security.jail.mount_nullfs_allowed: 1
security.jail.mount_fdescfs_allowed: 0
security.jail.mount_procfs_allowed: 0
security.jail.mount_zfs_allowed: 1
security.jail.mount_devfs_allowed: 1
security.jail.mount_fusefs_allowed: 0
security.jail.mount_tmpfs_allowed: 0
security.jail.param.sysvshm.: 0
security.jail.param.sysvsem.: 0
security.jail.param.sysvmsg.: 0
security.jail.param.allow.mount.nullfs: 0
security.jail.param.allow.mount.fdescfs: 0
security.jail.param.allow.mount.procfs: 0
security.jail.param.allow.mount.zfs: 0
security.jail.param.allow.mount.devfs: 0
security.jail.param.allow.mount.fusefs: 0
security.jail.param.allow.mount.tmpfs: 0
security.jail.param.allow.mount.: 0
security.jail.param.allow.suser: 0
security.jail.param.allow.unprivileged_proc_debug: 0
security.jail.param.allow.read_msgbuf: 0
security.jail.param.allow.reserved_ports: 0
security.jail.param.allow.mlock: 0
security.jail.param.allow.socket_af: 0
security.jail.param.allow.quotas: 0
security.jail.param.allow.chflags: 0
security.jail.param.allow.raw_sockets: 0
security.jail.param.allow.sysvipc: 0
security.jail.param.allow.set_hostname: 0
security.jail.param.ip6.saddrsel: 0
security.jail.param.ip6.: 0
security.jail.param.ip4.saddrsel: 0
security.jail.param.ip4.: 0
security.jail.param.cpuset.id: 0
security.jail.param.host.hostid: 0
security.jail.param.host.hostuuid: 64
security.jail.param.host.domainname: 256
security.jail.param.host.hostname: 256
security.jail.param.host.: 0
security.jail.param.children.max: 0
security.jail.param.children.cur: 0
security.jail.param.dying: 0
security.jail.param.vnet: 0
security.jail.param.persist: 0
security.jail.param.devfs_ruleset: 0
security.jail.param.enforce_statfs: 0
security.jail.param.osrelease: 32
security.jail.param.osreldate: 0
security.jail.param.securelevel: 0
security.jail.param.path: 1024
security.jail.param.name: 256
security.jail.param.parent: 0
security.jail.param.jid: 0
security.jail.devfs_ruleset: 0
security.jail.enforce_statfs: 2
security.jail.mount_allowed: 1
security.jail.chflags_allowed: 0
security.jail.allow_raw_sockets: 0
security.jail.sysvipc_allowed: 0
security.jail.socket_unixiproute_only: 1
security.jail.set_hostname_allowed: 1
security.jail.jail_max_af_ips: 255
security.jail.vnet: 0
security.jail.jailed: 0

Bastille jail conf:

file-server {
  devfs_ruleset = 4;
  enforce_statfs = 0;
  exec.clean;
  exec.consolelog = /var/log/bastille/file-server_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = file-server;
  #allow.raw_sockets = 1;
  allow.mount = 1;
  allow.mount.fdescfs = 1;
  allow.mount.fusefs = 1;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/file-server/fstab;
  path = /usr/local/bastille/jails/file-server/root;
  securelevel = 2;

  interface = genet0;
  ip4.addr = 10.7.4.101;
  ip6 = disable;

wsdd runs fine on the host.

christgau commented 1 year ago

I was able to reproduce the issue with your jail configuration (but slightly different paths and IPs which does not matter).

christgau commented 1 year ago

I posted a description of the underlying problem to the freebsd-net mailing list. I have the impression that either wsdd is doing something wrong or there is a flaw in how FreeBSD handles binding of multicast sockets in jails. The former might be more likely than the latter 😉 but I don't see how this can be fixed in a neat way.

christgau commented 1 year ago

It was pointed out, that VNET-style jails may work as a solution here. I was able to setup such a jail in a FreeBSD VM (here is a tutorial, that may help along with the link from the mailing list) and wsdd starts fine. However, the host is not announced to my Windows machine that hosts the VM, but it may be due to incomplete IP setup. Does that approach works for you?

christgau commented 1 year ago

It was further pointed out at the mailing list, that there is no multicast support in "classic plaing old IP jails" and VNET jails are the way to go. So it appears to me, that there is nothing that can be done from a wsdd perspective. I therefore close that issue, but feel free to share your experiences/findings/configuration here in case you get a working solution.