search

tailscale-dev / docker-mod

The home for our universal Docker mod
BSD 3-Clause "New" or "Revised" License
63 stars 30 forks source link

Some containers fail to bind the specified port #18

Open noahkiss opened 10 months ago

noahkiss commented 10 months ago

I've seen this issue with at least two lsio containers: heimdall & nextcloud.

I use the same compose file with all of my lsio containers, which works with most of them (around 15 with the tailscale mod), with only ports & names changed, generally.

My understanding of what's happening is that Tailscale is creating its own server on 443 (or maybe 80 if you select http) and then forwarding requests to the internal container's port (ex. radarr, it's going from 443 -> 7878). But any containers that internally already use 80 or 443 cause a port conflict because they are also creating listeners on the ports that tailscale has already registered (since tailscale creates during the S6 bootup).

In a perfect world, there would be a way to allow both to exist simultaneously - I'm not sure if that would mean allowing Tailscale to create the listener on non-standard ports, or somehow creating a reverse proxy and then remapping the container's ports with an internal port mapping option similar to the standard compose port map, or what.

Docker Compose ```yaml --- version: "3" services: heimdall: image: linuxserver/heimdall:latest container_name: heimdall cap_add: - NET_ADMIN - NET_RAW environment: # Tailscale TAILSCALE_HOSTNAME: heimdall TAILSCALE_SERVE_PORT: 443 # or 80 DOCKER_MODS: ghcr.io/tailscale-dev/docker-mod:main # Environment PUID: 1050 PGID: 100 TZ: America/New_York # Tailscale TAILSCALE_STATE_DIR: /var/lib/tailscale TAILSCALE_SERVE_MODE: https TAILSCALE_USE_SSH: 1 FILE__TAILSCALE_AUTHKEY: /run/secrets/tailscale_authkey ports: - 2800:80 - 2801:443 volumes: - /volume1/docker/containers/heimdall/config:/config # Tailscale - /dev/net/tun:/dev/net/tun - /volume1/docker/secrets/tailscale_authkey:/run/secrets/tailscale_authkey - tailscale:/var/lib/tailscale restart: on-failure:5 volumes: tailscale: ```
Heimdall Log ```js stdout [mod-init] Adding tailscale-dev/docker-mod:main to container stdout [mod-init] Downloading tailscale-dev/docker-mod:main from ghcr.io stdout [mod-init] Installing tailscale-dev/docker-mod:main stdout [mod-init] tailscale-dev/docker-mod:main applied to container stdout [migrations] started stdout [migrations] 01-nginx-site-confs-default: skipped stdout [migrations] 02-default-location: skipped stdout [migrations] done stdout [env-init] TAILSCALE_AUTHKEY set from FILE__TAILSCALE_AUTHKEY stdout ─────────────────────────────────────── stdout stdout ██╗ ███████╗██╗ ██████╗ stdout ██║ ██╔════╝██║██╔═══██╗ stdout ██║ ███████╗██║██║ ██║ stdout ██║ ╚════██║██║██║ ██║ stdout ███████╗███████║██║╚██████╔╝ stdout ╚══════╝╚══════╝╚═╝ ╚═════╝ stdout stdout Brought to you by linuxserver.io stdout ─────────────────────────────────────── stdout stdout To support LSIO projects visit: stdout https://www.linuxserver.io/donate/ stdout stdout ─────────────────────────────────────── stdout GID/UID stdout ─────────────────────────────────────── stdout stdout User UID: 1050 stdout User GID: 100 stdout ─────────────────────────────────────── stdout stdout using keys found in /config/keys stdout New container detected, installing Heimdall stdout [mod-init] **** Installing all mod packages **** stdout fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz stdout fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz stdout (1/3) Installing libmnl (1.0.5-r1) stdout (2/3) Installing libnftnl (1.2.5-r1) stdout (3/3) Installing iptables (1.8.9-r2) stdout Executing busybox-1.36.1-r5.trigger stdout OK: 75 MiB in 94 packages stdout /run/s6-rc:s6-rc-init:hjgImF/servicedirs/s6rc-oneshot-runner /run/s6-rc:s6-rc-init:hjgImF/servicedirs/s6rc-oneshot-runner stderr % Total % Received % Xferd Average Speed Time Time Time Current stderr Dload Upload Total Spent Left Speed stderr 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2820 100 2820 0 0 14654 # [...] stderr % Total % Received % Xferd Average Speed Time Time Time Current stderr Dload Upload Total Spent Left Speed stderr 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 29.4M 0 256k 0 0 498k # [...] stdout 'tailscale_1.56.1_amd64/tailscale' -> '/usr/bin/tailscale' stdout 'tailscale_1.56.1_amd64/tailscaled' -> '/usr/sbin/tailscaled' stdout /run/s6-rc:s6-rc-init:hjgImF/servicedirs/s6rc-oneshot-runner stdout 2024/01/12 10:41:06 logtail started stdout 2024/01/12 10:41:06 Program starting: v1.56.1-t906f85d10-g34ed54c8c, Go 1.21.5: []string{"tailscaled", "--statedir=/var/lib/tailscale"} stdout 2024/01/12 10:41:06 LogID: 854c0b6eea497981c32f74ddbc0ff9d63965f7c1585370e75c48047274c963c77 stdout 2024/01/12 10:41:06 logpolicy: using system state directory "/var/lib/tailscale" stdout logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory stdout logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil stdout 2024/01/12 10:41:06 wgengine.NewUserspaceEngine(tun "tailscale0") ... stdout 2024/01/12 10:41:06 setting link attributes: setsockopt: protocol not available stdout 2024/01/12 10:41:06 router: failed to determine ip command fwmask support: exit status 1 stdout 2024/01/12 10:41:06 dns: [rc=unknown ret=direct] stdout 2024/01/12 10:41:06 dns: using "direct" mode stdout 2024/01/12 10:41:06 dns: using *dns.directManager stdout 2024/01/12 10:41:06 link state: interfaces.State{defaultRoute=eth0 ifs={eth0:[192.168.130.2/20]} v4=true v6=false} stdout 2024/01/12 10:41:06 onPortUpdate(port=56705, network=udp6) stdout 2024/01/12 10:41:06 router: using firewall mode pref stdout 2024/01/12 10:41:06 router: default choosing iptables stdout 2024/01/12 10:41:06 router: disabling tunneled IPv6 due to system IPv6 config: disable_ipv6 is set stdout 2024/01/12 10:41:06 onPortUpdate(port=57625, network=udp4) stdout 2024/01/12 10:41:06 magicsock: disco key = d:5111660f1f4f9ade stdout 2024/01/12 10:41:06 Creating WireGuard device... stdout 2024/01/12 10:41:06 Bringing WireGuard device up... stdout 2024/01/12 10:41:06 external route: up stdout 2024/01/12 10:41:06 Bringing router up... stdout 2024/01/12 10:41:06 Clearing router settings... stdout 2024/01/12 10:41:06 Starting network monitor... stdout 2024/01/12 10:41:06 Engine created. stdout 2024/01/12 10:41:06 pm: migrating "_daemon" profile to new format stdout 2024/01/12 10:41:06 logpolicy: using system state directory "/var/lib/tailscale" stdout 2024/01/12 10:41:06 got LocalBackend in 590ms stdout 2024/01/12 10:41:06 Start stdout 2024/01/12 10:41:06 Backend: logs: be:854c0b6eea497981c32f74ddbc0ff9d63965f7c1585370e75c48047274c963c77 fe: stdout 2024/01/12 10:41:06 Switching ipn state NoState -> NeedsLogin (WantRunning=false, nm=false) stdout 2024/01/12 10:41:06 blockEngineUpdates(true) stdout 2024/01/12 10:41:06 health("overall"): error: state=NeedsLogin, wantRunning=false stdout 2024/01/12 10:41:06 wgengine: Reconfig: configuring userspace WireGuard config (with 0/0 peers) stdout 2024/01/12 10:41:06 wgengine: Reconfig: configuring router stdout 2024/01/12 10:41:06 wgengine: Reconfig: configuring DNS stdout 2024/01/12 10:41:06 dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:0} stdout 2024/01/12 10:41:06 dns: Resolvercfg: {Routes:{} Hosts:0 LocalDomains:[]} stdout 2024/01/12 10:41:06 dns: OScfg: {} stdout 2024/01/12 10:41:06 Start stdout 2024/01/12 10:41:06 generating new machine key stdout 2024/01/12 10:41:07 machine key written to store stdout 2024/01/12 10:41:07 control: client.Shutdown() stdout 2024/01/12 10:41:07 control: client.Shutdown stdout 2024/01/12 10:41:07 control: mapRoutine: exiting stdout 2024/01/12 10:41:07 control: authRoutine: exiting stdout 2024/01/12 10:41:07 control: updateRoutine: exiting stdout 2024/01/12 10:41:07 control: Client.Shutdown done. stdout 2024/01/12 10:41:07 Backend: logs: be:854c0b6eea497981c32f74ddbc0ff9d63965f7c1585370e75c48047274c963c77 fe: stdout 2024/01/12 10:41:07 Switching ipn state NoState -> NeedsLogin (WantRunning=true, nm=false) stdout 2024/01/12 10:41:07 blockEngineUpdates(true) stdout 2024/01/12 10:41:07 StartLoginInteractive: url=false stdout 2024/01/12 10:41:07 control: client.Login(false, 2) stdout 2024/01/12 10:41:07 control: LoginInteractive -> regen=true stdout 2024/01/12 10:41:07 control: doLogin(regen=true, hasUrl=false) stdout 2024/01/12 10:41:08 control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp] stdout 2024/01/12 10:41:08 control: Generating a new nodekey. stdout 2024/01/12 10:41:08 control: RegisterReq: onode= node=[HXvXs] fup=false nks=false stdout 2024/01/12 10:41:09 control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false stdout 2024/01/12 10:41:09 blockEngineUpdates(false) stdout 2024/01/12 10:41:09 control: netmap: got new dial plan from control stdout 2024/01/12 10:41:11 active login: noahkiss@github stdout 2024/01/12 10:41:11 monitor: gateway and self IP changed: gw=192.168.130.1 self=192.168.130.2 stdout 2024/01/12 10:41:12 control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=false udp=true icmpv4=false derp=#1 portmap= link="" firewallmode="ipt-default"} stdout 2024/01/12 10:41:12 magicsock: endpoints changed: [External IP]:57625 (stun), 192.168.130.2:57625 (local) stdout 2024/01/12 10:41:12 Switching ipn state NeedsLogin -> Starting (WantRunning=true, nm=true) stdout 2024/01/12 10:41:12 magicsock: SetPrivateKey called (init) stdout 2024/01/12 10:41:12 magicsock: private key changed, reconnecting to home derp-1 stdout 2024/01/12 10:41:12 magicsock: adding connection to derp-1 for home-keep-alive stdout 2024/01/12 10:41:12 magicsock: 1 active derp conns: derp-1=cr0s,wr0s stdout 2024/01/12 10:41:12 derphttp.Client.Connect: connecting to derp-1 (nyc) stdout 2024/01/12 10:41:12 wgengine: Reconfig: configuring userspace WireGuard config (with 0/19 peers) stdout 2024/01/12 10:41:12 wgengine: Reconfig: configuring router stdout 2024/01/12 10:41:12 magicsock: derp-1 connected; connGen=1 stdout 2024/01/12 10:41:12 health("overall"): ok stdout 2024/01/12 10:41:12 wgengine: Reconfig: configuring DNS stdout 2024/01/12 10:41:12 dns: Set: {DefaultResolvers:[https://dns.nextdns.io/XXXXXX?device_id=nTctNJvbma11CNTRL&device_name=heimdall&device_model=linux&device_ip=100.103.134.11] Routes:{peter-piper.ts.net.:[] ts.net.:[188.257.153.40 2620:111:8007::53]}+65arpa SearchDomains:[peter-piper.ts.net.] Hosts:20} stdout 2024/01/12 10:41:12 dns: Resolvercfg: {Routes:{.:[https://dns.nextdns.io/XXXXXX?device_id=nTctNJvbmqa11CNTRL&device_name=heimdall&device_model=linux&device_ip=100.103.134.11] ts.net.:[188.257.153.40 2620:111:8007::53]} Hosts:20 LocalDomains:[peter-piper.ts.net.]+65arpa} stdout 2024/01/12 10:41:12 dns: OScfg: {Nameservers:[100.100.100.100] SearchDomains:[peter-piper.ts.net.] } stdout 2024/01/12 10:41:12 rename of "/etc/resolv.conf" to "/etc/resolv.pre-tailscale-backup.conf" failed (rename /etc/resolv.conf /etc/resolv.pre-tailscale-backup.conf: device or resource busy), falling back to copy+delete stdout 2024/01/12 10:41:12 peerapi: serving on http://100.103.134.11:36888 stdout 2024/01/12 10:41:12 peerapi: failed to do peerAPI listen, harmless (netstack available) but error was: listen tcp6 [fd7a:115c:a1e0::a3f8:900b]:0: bind: cannot assign requested address stdout 2024/01/12 10:41:12 peerapi: serving on http://[fd7a:115c:a1e0::a4a8:900b]:1 stdout 2024/01/12 10:41:12 Switching ipn state Starting -> Running (WantRunning=true, nm=true) stdout 2024/01/12 10:41:13 serve: creating a new proxy handler for http://127.0.0.1:80 stdout 2024/01/12 10:41:13 listening on 100.103.134.11:443 stdout Available within your tailnet: stdout stdout https://heimdall.peter-piper.ts.net/ stdout |-- proxy http://127.0.0.1:80 stdout stdout Serve started and running in the background. stdout To disable the proxy, run: tailscale serve --https=443 off stdout [custom-init] No custom files found, skipping... stdout [ls.io-init] done. stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) stderr nginx: [emerg] still could not bind() stdout 2024/01/12 10:42:15 wgengine: idle peer [TS7Bg] now active, reconfiguring WireGuard stdout 2024/01/12 10:42:15 wgengine: Reconfig: configuring userspace WireGuard config (with 1/19 peers) stdout 2024/01/12 10:42:15 magicsock: disco: node [TS7Bg] d:0ccd0253da897f39 now using 192.168.5.24:41641 mtu=1360 tx=c7108e9c66dd stdout 2024/01/12 10:42:16 Accept: TCP{100.118.64.70:54629 > 100.103.134.11:443} 64 tcp ok stdout 2024/01/12 10:42:16 Accept: TCP{100.118.64.70:54629 > 100.103.134.11:443} 52 tcp non-syn stdout 2024/01/12 10:42:16 Accept: TCP{100.118.64.70:54629 > 100.103.134.11:443} 569 tcp non-syn stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) stderr nginx: [emerg] still could not bind() stdout 2024/01/12 10:42:17 cert("heimdall.peter-piper.ts.net"): registered ACME account. stdout 2024/01/12 10:42:17 cert("heimdall.peter-piper.ts.net"): starting SetDNS call... stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) stderr nginx: [emerg] still could not bind() stdout 2024/01/12 10:42:28 cert("heimdall.peter-piper.ts.net"): did SetDNS stdout 2024/01/12 10:42:30 cert("heimdall.peter-piper.ts.net"): requesting cert... stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) stderr nginx: [emerg] still could not bind() stdout 2024/01/12 10:42:31 cert("heimdall.peter-piper.ts.net"): got cert stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) stdout 2024/01/12 10:42:32 Accept: TCP{100.118.64.70:54629 > 100.103.134.11:443} 52 tcp non-syn stdout 2024/01/12 10:42:35 http: proxy error: read tcp 127.0.0.1:34550->127.0.0.1:80: read: connection reset by peer stdout 2024/01/12 10:42:35 http: proxy error: dial tcp 127.0.0.1:80: connect: connection refused stderr nginx: [emerg] still could not bind() stderr nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address in use) ```
noahkiss commented 10 months ago

Maybe the answer is to add another configuration variable that could set a non-standard listener port?

For example, adding TAILSCALE_HOST_PORT:

Environment Variable Description Example
TAILSCALE_SERVE_PORT The (internal/container) port number that you want to expose on your tailnet. This will be the port that your DokuWiki, Transmission, or other container is internally listening on. 80
TAILSCALE_HOST_PORT The external port number that you want to listen on inside your container. If your container already listens on 80 and/or 443, this avoids conflicts. For example, setting the value to 8443 would allow you to access your container at https://wiki.yourtailnet.ts.net:8443. 443 
# configure serve or funnel
if [ -v TAILSCALE_SERVE_PORT ] && [ -v TAILSCALE_SERVE_MODE ]; then

    if [ -v TAILSCALE_FUNNEL ]; then
        tailscale funnel --bg \
          --"${TAILSCALE_SERVE_MODE}"=${TAILSCALE_HOST_PORT:-443} http://localhost:"${TAILSCALE_SERVE_PORT}"
    else
        tailscale serve --bg \
          --"${TAILSCALE_SERVE_MODE}"=${TAILSCALE_HOST_PORT:-443} http://localhost:"${TAILSCALE_SERVE_PORT}"
    fi

fi