search

qdm12 / gluetun

VPN client in a thin Docker container for multiple VPN providers, written in Go, and using OpenVPN or Wireguard, DNS over TLS, with a few proxy servers built-in.
https://hub.docker.com/r/qmcgaw/gluetun
MIT License
7.68k stars 359 forks source link

Bug: Custom Wireguard cannot reach other containers #1224

Open schklom opened 1 year ago

schklom commented 1 year ago

Is this urgent?

No

Host OS

Debian Bullseye

CPU arch

aarch64

VPN service provider

Custom

What are you using to run the container

docker-compose

What is the version of Gluetun

Running version latest built on 2022-10-31T11:25:29.199Z (commit 192a7a5)

What's the problem 🤔

gluetun cannot retrieve data from containers on the same network despite specifying FIREWALL_OUTBOUND_SUBNETS correctly

$ docker exec vpn wget "http://172.18.58.15"
Connecting to 172.18.58.15 (172.18.58.15:80)
wget: can't connect to remote host (172.18.58.15): Operation timed out

but these containers work well

$ curl "http://127.0.0.1:8000"
Hostname: whoami
IP: 127.0.0.1
IP: 172.18.58.15
RemoteAddr: 10.0.2.2:49570
GET / HTTP/1.1
Host: 127.0.0.1:8000
User-Agent: curl/7.74.0
Accept: */*

Any idea what I am doing wrong?

Share your logs

$ docker logs vpn
========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2022-10-31T11:25:29.199Z (commit 192a7a5)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? quentin.mcgaw@gmail.com
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2022-11-05T14:17:25+01:00 INFO [routing] default route found: interface eth0, gateway 172.18.58.1 and assigned IP 172.18.58.2
2022-11-05T14:17:25+01:00 INFO [routing] local ethernet link found: eth0
2022-11-05T14:17:25+01:00 INFO [routing] local ipnet found: 172.18.58.0/24
2022-11-05T14:17:25+01:00 INFO [firewall] enabling...
2022-11-05T14:17:25+01:00 INFO [firewall] enabled successfully
2022-11-05T14:17:26+01:00 INFO [storage] creating /gluetun/servers.json with 13220 hardcoded servers
2022-11-05T14:17:27+01:00 INFO Alpine version: 3.16.2
2022-11-05T14:17:27+01:00 INFO OpenVPN 2.4 version: 2.4.12
2022-11-05T14:17:27+01:00 INFO OpenVPN 2.5 version: 2.5.6
2022-11-05T14:17:27+01:00 INFO Unbound version: 1.15.0
2022-11-05T14:17:27+01:00 INFO IPtables version: v1.8.8
2022-11-05T14:17:27+01:00 INFO Settings summary:
├── VPN settings:
|   ├── VPN provider settings:
|   |   ├── Name: custom
|   |   └── Server selection settings:
|   |       ├── VPN type: wireguard
|   |       ├── Target IP address: X.X.X.X
|   |       └── Wireguard selection settings:
|   |           ├── Endpoint IP address: X.X.X.X
|   |           ├── Endpoint port: 51820
|   |           └── Server public key: XXXX
|   └── Wireguard settings:
|       ├── Private key: EN...HE=
|       ├── Interface addresses:
|       |   └── 10.2.0.2/32
|       └── Network interface: tun0
├── DNS settings:
|   ├── DNS server address to use: 127.0.0.1
|   ├── Keep existing nameserver(s): no
|   └── DNS over TLS settings:
|       ├── Enabled: yes
|       ├── Update period: every 24h0m0s
|       ├── Unbound settings:
|       |   ├── Authoritative servers:
|       |   |   └── quad9
|       |   ├── Caching: yes
|       |   ├── IPv6: no
|       |   ├── Verbosity level: 1
|       |   ├── Verbosity details level: 0
|       |   ├── Validation log level: 0
|       |   ├── System user: root
|       |   └── Allowed networks:
|       |       ├── 0.0.0.0/0
|       |       └── ::/0
|       └── DNS filtering settings:
|           ├── Block malicious: yes
|           ├── Block ads: no
|           ├── Block surveillance: no
|           └── Blocked IP networks:
|               ├── 127.0.0.1/8
|               ├── 10.0.0.0/8
|               ├── 172.16.0.0/12
|               ├── 192.168.0.0/16
|               ├── 169.254.0.0/16
|               ├── ::1/128
|               ├── fc00::/7
|               ├── fe80::/10
|               ├── ::ffff:7f00:1/104
|               ├── ::ffff:a00:0/104
|               ├── ::ffff:a9fe:0/112
|               ├── ::ffff:ac10:0/108
|               └── ::ffff:c0a8:0/112
├── Firewall settings:
|   ├── Enabled: yes
|   └── Outbound subnets:
|       └── {172.18.0.0 ffff0000}
├── Log settings:
|   └── Log level: INFO
├── Health settings:
|   ├── Server listening address: 127.0.0.1:9999
|   ├── Target address: cloudflare.com:443
|   ├── Read header timeout: 100ms
|   ├── Read timeout: 500ms
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   └── Enabled: no
├── HTTP proxy settings:
|   └── Enabled: no
├── Control server settings:
|   ├── Listening address: :8000
|   └── Logging: yes
├── OS Alpine settings:
|   ├── Process UID: 1000
|   ├── Process GID: 1000
|   └── Timezone: Europe/Oslo
├── Public IP settings:
|   ├── Fetching: every 12h0m0s
|   └── IP file path: /tmp/gluetun/ip
└── Version settings:
    └── Enabled: yes
2022-11-05T14:17:27+01:00 INFO IPv6 is not supported
2022-11-05T14:17:27+01:00 INFO [routing] default route found: interface eth0, gateway 172.18.58.1 and assigned IP 172.18.58.2
2022-11-05T14:17:27+01:00 INFO [routing] adding route for 0.0.0.0/0
2022-11-05T14:17:27+01:00 INFO [firewall] setting allowed subnets...
2022-11-05T14:17:27+01:00 INFO [routing] default route found: interface eth0, gateway 172.18.58.1 and assigned IP 172.18.58.2
2022-11-05T14:17:27+01:00 INFO [routing] adding route for 172.18.0.0/16
2022-11-05T14:17:27+01:00 INFO [pprof] http server listening on [::]:6060
2022-11-05T14:17:27+01:00 INFO [dns over tls] using plaintext DNS at address 9.9.9.9
2022-11-05T14:17:27+01:00 INFO [http server] http server listening on [::]:8000
2022-11-05T14:17:27+01:00 INFO [firewall] allowing VPN connection...
2022-11-05T14:17:27+01:00 INFO [healthcheck] listening on 127.0.0.1:9999
2022-11-05T14:17:27+01:00 INFO [wireguard] Using available kernelspace implementation
2022-11-05T14:17:27+01:00 INFO [wireguard] Connecting to X.X.X.X:51820
2022-11-05T14:17:27+01:00 INFO [wireguard] Wireguard is up
2022-11-05T14:17:27+01:00 INFO [dns over tls] downloading DNS over TLS cryptographic files
2022-11-05T14:17:28+01:00 INFO [healthcheck] healthy!
2022-11-05T14:17:29+01:00 INFO [dns over tls] downloading hostnames and IP block lists
2022-11-05T14:17:41+01:00 INFO [healthcheck] unhealthy: cannot dial: dial tcp4: lookup cloudflare.com: i/o timeout
2022-11-05T14:17:44+01:00 INFO [dns over tls] init module 0: validator
2022-11-05T14:17:44+01:00 INFO [dns over tls] init module 1: iterator
2022-11-05T14:17:44+01:00 INFO [dns over tls] start of service (unbound 1.15.0).
2022-11-05T14:17:44+01:00 INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
2022-11-05T14:17:44+01:00 INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
2022-11-05T14:17:45+01:00 INFO [healthcheck] healthy!
2022-11-05T14:17:45+01:00 INFO [dns over tls] ready
2022-11-05T14:17:45+01:00 INFO [vpn] You are running on the bleeding edge of latest!
2022-11-05T14:17:45+01:00 INFO [ip getter] Public IP address is X.X.X.X (Netherlands, XXXX)

Share your configuration

services:
  vpn:
    image: qmcgaw/gluetun
    container_name: vpn
    hostname: vpn
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun # Requires access to TUN devices with Rootless Docker
    environment:
      TZ: ${TZ}

      # DNS
      # https://github.com/qdm12/gluetun/wiki/DNS-options
      DOT_PROVIDERS: quad9

      # LAN ACCESS
      # https://github.com/qdm12/gluetun/wiki/Firewall-options
      FIREWALL_OUTBOUND_SUBNETS: 172.18.0.0/16

      VPN_SERVICE_PROVIDER: custom
      VPN_TYPE: wireguard

      # For Wireguard
      VPN_ENDPOINT_IP: X.X.X.X
      VPN_ENDPOINT_PORT: 51820
      WIREGUARD_PUBLIC_KEY: XXXX
      WIREGUARD_PRIVATE_KEY: XXXX
      WIREGUARD_ADDRESSES: 10.2.0.2/32

      # https://github.com/qdm12/gluetun/wiki/HTTP-proxy-options
      HTTPPROXY: "on"
      HTTPPROXY_USER: XXXX
      HTTPPROXY_PASSWORD: XXXX

    ports:
      - 8888:8888 # HTTP Proxy
    networks:
      vpn_net:

  whoami:
    image: containous/whoami
    container_name: whoami
    hostname: whoami
    ports:
      - 8000:80
    networks:
      vpn_net:
        ipv4_address: 172.18.58.15

networks:
  vpn_net:
    ipam:
      config:
        - subnet: 172.18.58.0/24

Edit: replaced the network 192.168.0.0/24 with 172.18.58.0/24, but gluetun still refuses to connect. Edit2: Removing 192.168.0.0/24 from DOT_PRIVATE_ADDRESS and adding it to FIREWALL_OUTBOUND_SUBNETS does not help either.

bnhf commented 1 year ago

@schklom

Maybe you're trying to accomplish something that I'm missing, but your stack should look more like this -- whether you're using WireGuard or OpenVPN:

version: '3.7'
services:
    gluetun:
        image: qmcgaw/gluetun:latest
        container_name: gluetun
        cap_add:
            - NET_ADMIN
        ports:
            - 8100:8000 # Remote Control VPN
            - 3000:3000 # Firefox
        environment:
            - VPN_SERVICE_PROVIDER=airvpn
            - VPN_TYPE=openvpn
            - SERVER_COUNTRIES=Canada
            - OPENVPN_CIPHERS=AES-256-GCM
            - PUID=1000
            - PGID=1000
            - TZ=US/Mountain
        volumes:
            - /data/openvpn:/gluetun # /data/openvpn contains client.crt and client.key

    firefox:
        image: lscr.io/linuxserver/firefox:latest
        container_name: firefox
        environment:
            - PUID=1000
            - PGID=1000
            - TZ=US/Mountain
        volumes:
            - /data/firefox:/config
        shm_size: '1gb'
        network_mode: 'service:gluetun'
        depends_on:
            - gluetun

Any open ports are defined in the Gluetun section, regardless of which container needs them. Additional containers you want in the stack use a network mode of 'service:gluetun'. Your dependent containers are not assigned their own Docker subnet IP, but rather all use the IP assigned to Gluetun. The only path to the Internet for these containers is through the VPN.

schklom commented 1 year ago

@bnhf Thank you for the reply, but I do not want my other container to be unreachable if gluetun goes down. In your setup, if gluetun goes down then firefox is not reachable anymore.

My setup is not complex, I simply want gluetun on the same network as whoami and be able to reach it.

bnhf commented 1 year ago

@schklom

When you say "if gluetun goes down", do you mean if the VPN goes down? In my experience, regardless of whether the VPN is connected or not, the containers linked through Gluetun are still accessible from the local network.

schklom commented 1 year ago

@bnhf I guess we have a different experience. For me, rarely, the container crashes. Restarting it makes everything work fine again though. If I do it your way, I will also have to worry about port collisions, because many containers use port 80 internally.

I just want the containers to be accessible without forcing them all through a container's network.

bnhf commented 1 year ago

@schklom

Gluetun container crashes are something that @qdm12 might be interested to know more about.

I hear what you're saying about unresolvable port collisions. Not something I've run into myself -- but maybe that's just luck. :-)

I came across a step-by-step, from the good people over at linuxserver.io, that describes setting up a Docker-based WireGuard client in the manner you'd like. Not sure if the method they're describing would lend itself to Gluetun, but it is an interesting read on what's required to route host and/or container traffic through a WireGuard container -- via your desired approach.

https://www.linuxserver.io/blog/routing-docker-host-and-container-traffic-through-wireguard