expose external port? - Githubissues

gingerlime commented 3 years ago

Hi there! Thanks for creating docker-wireguard, it looks great!

I have a question about exposing ports.

Additionally, you can expose ports to allow your local network to access services linked to the Wireguard container:

docker run --name wireguard                                          \
    --cap-add NET_ADMIN                                              \
    --cap-add SYS_MODULE                                             \
    --sysctl net.ipv4.conf.all.src_valid_mark=1                      \
    -v /path/to/conf/mullvad.conf:/etc/wireguard/mullvad.conf        \
    -p 8080:80                                                       \
    jordanpotter/wireguard

docker run -it --rm                                                  \
    --net=container:wireguard                                        \
    nginx

The documentation is very clear, and it works locally, so after running this example, I can do

# curl http://localhost:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

So this works great! However, I'm wondering about exposing this port to the outside world. If I run this on a remote server, even though I can see that the server is listening on 0.0.0.0, I cannot remotely connect ...

On the server where I run docker-wireguard and nginx:

# netstat -tulpn |grep 8080
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      19255/docker-proxy
tcp6       0      0 :::8080                 :::*                    LISTEN      19260/docker-proxy

On a remote client:

curl http://{remote-host}:8080
curl: (7) Failed to connect to {remote-host} port 8080: Operation timed out

Any tips on how to expose ports of services to the outside, and yet let these services communicate to the outside world via the wireguard link?

jordanpotter commented 3 years ago

Hey @gingerlime, thanks for submitting this issue! Sorry for the late reply -- things have been a bit hectic lately.

On chance are there firewall rules on the remote host preventing it from receiving traffic on port 8080? An easy way to check is to turn off the Docker container, run nc -l -p 8080 on the remote host, then curl http://{remote-host}:8080 from your client.

Let me know what you discover!

gingerlime commented 3 years ago

@jordanpotter thank you. I thought docker (or docker-compose) pokes holes in firewalls by default? but I also tested it without a firewall. nc works, but using the docker setup I posted above does not.

I managed to work around it using traefik, but I still wonder why it doesn't work out-of-the-box?

jordanpotter commented 3 years ago

Hrmm, very curious. I'll do a bit of experimentation over the weekend and get back to you.

jordanpotter commented 2 years ago

Hey @gingerlime, could you try adding the LOCAL_SUBNET environment variable when running the Wireguard container?

In my test setup, I have two machines on a 10.0.0.0/8 network. Let's say machine A has IP address 10.0.0.1 and machine B has IP address 10.0.0.2.

On machine A, I started the Wireguard and Nginx containers the exact same as you did. Curling 127.0.0.1:8080 on machine A worked, however curling 10.0.0.1:8080 on machine B failed. The reason is that the killswitch in the Wireguard container is forbidding traffic to the 10.0.0.0/8 network, so machine B on 10.0.0.2 will never receive a response.

I then modified the docker run command for the Wireguard container by adding the LOCAL_SUBNET environment variable:

docker run --name wireguard                                          \
    --cap-add NET_ADMIN                                              \
    --cap-add SYS_MODULE                                             \
    --sysctl net.ipv4.conf.all.src_valid_mark=1                      \
    -v /path/to/conf/mullvad.conf:/etc/wireguard/mullvad.conf        \
    -p 8080:80                                                       \
    -e LOCAL_SUBNET=10.0.0.0/8                                       \ 
    jordanpotter/wireguard

Afterwards, curling 10.0.0.1:8080 on machine B worked correctly.

If this works for you too, I'd like to update the documentation to make this clearer 👍

gingerlime commented 2 years ago

Thanks @jordanpotter ... I'm not sure I fully understand. I have only one machine, and I'm not sure I have those local subnets configured... I want to curl the public IP address... Plus, it works with traefik out of the box, so I wonder what they do to make it work?

jordanpotter commented 2 years ago

Oh, I misunderstood the problem! Sorry for the confusion.

The Wireguard container has a killswitch that requires all traffic to be (1) over Wireguard, (2) to the local network, or (3) to the Docker network. So when you curl from your local machine to the Wireguard container running on your remote server, the killswitch will prevent Nginx from responding to the request.

However when you run Traefik, Traefik is proxying requests to the Wireguard container over the Docker network (which the killswitch allows). This is why everything is working with Traefik 👍

If we want this to work without Traefik, then the Wireguard container needs to know which ports to allow in the killswitch. Similar to what we do with LOCAL_SUBNET. Unfortunately it appears not possible to automatically detect the exposed ports from inside the container, so we'd likely need another environment variable here.

Or we can just recommend people use Traefik 😄

gingerlime commented 2 years ago

Hi @jordanpotter. Thanks for the patience and taking the time to explain :) I really appreciate it.

Or we can just recommend people use Traefik 😄

Yes, sounds like a good idea. Here's an example that I hope can be helpful for others. It has a few specific parts (e.g. using letsencrypt with cloudflare), but otherwise I tried to add comments to explain how things fit together the best I could (I'm no expert on traefik, it's the first time I used it, and I hope I didn't make any glaring mistakes, because it's quite complex and feature-rich).

version: '3'
services:
  traefik:
    image: traefik:v2.4
    container_name: traefik
    command:
      - --api.insecure=true
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.websecure.address=:443
      # Set up LetsEncrypt (in this example, via cloudflare)
      - --certificatesresolvers.letsencrypt.acme.dnschallenge=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
      - --entrypoints.websecure.http.tls=true
      - --entrypoints.websecure.http.tls.certResolver=letsencrypt
      # let's say our domain is example.com and we want to serve apps on subdomains
      # e.g. proxy.example.com, app.example.com etc
      - --entrypoints.websecure.http.tls.domains[0].main=example.com
      - --entrypoints.websecure.http.tls.domains[0].sans=*.example.com
      # Define the apps/subdomains, each with its own unique port
      # In this example we use two endpoints:
      #   - app.example.com is an HTTP app
      #   - proxy.example.com is an app that listens on a TCP port (e.g. SOCKS5)
      - --entrypoints.proxy.address=:8900
      - --entrypoints.app.address=:8090
    ports:
      - 80:80
      - 443:443
      # besides mapping HTTP 80 + 443, we map ports for each app
      - 8900:8900
      - 8090:8090
    environment:
      # if you're using Cloudflare, you would need the API TOKEN as an environment variable
      - CF_DNS_API_TOKEN
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # create a folder to store the letsencrypt data
      - /path/to/letsencrypt:/letsencrypt
    restart: unless-stopped
  wireguard:
    container_name: wireguard
    image: jordanpotter/wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    sysctls:
      net.ipv4.conf.all.src_valid_mark: 1
    volumes:
      - /path/to/conf/mullvad.conf:/etc/wireguard/mullvad.conf
    restart: unless-stopped
  app:
    # image: your-app
    network_mode: service:wireguard
    depends_on:
      - wireguard
    labels:
      # in this example, we have an HTTP app using the name `app` on app.example.com
      - traefik.enable=true
      - traefik.http.services.app.loadbalancer.server.port=8090
      - traefik.http.routers.app.rule=Host(`app.example.com`)
      - traefik.http.routers.app.entrypoints=websecure
      - traefik.http.routers.app.tls.certresolver=letsencrypt
    restart: unless-stopped
  proxy:
    # image: your-proxy-app
    network_mode: service:wireguard
    depends_on:
      - wireguard
    labels:
      # these labels link this container to traefik. Note to use the same port
      # this app listens on a (non-HTTP) port, so the configuration is slightly
      # different from our HTTP app; the app name is `proxy`
      - traefik.enable=true
      - traefik.tcp.routers.proxy.rule=HostSNI(`*`)
      - traefik.tcp.services.proxy.loadbalancer.server.port=8900
      - traefik.tcp.routers.proxy.entrypoints=proxy

5arer commented 1 year ago

Hi @jordanpotter I've tried to make the new environment variable and expose ports using iptables, and wanted to make the change in the image to fix this issue. However, I'm not sure I'm doing it well as I'm not able to make it work.

I've tried to put this after the LOCAL_SUBNETS logic that is allowing subnets

sudo iptables -I OUTPUT -p tcp --dport 6000 -j ACCEPT

then I've tried both INPUT and FORWARD but still not able to make it work I guess there is more to it? Any idea?

Thanks in advance

jordanpotter commented 1 year ago

Hey @5arer , to clarify, could you provide more details on your setup? Is the objective to (1) expose a port to your local subnet, (2) to the internet, or (3) to the internet over the WireGuard connection?

In my testing with Docker and Podman, for local access, simply exposing a port on the WireGuard container works. In fact, in CI I added a test that does exactly that: https://github.com/jordanpotter/docker-wireguard/blob/5c07117e1d87cc52dd7db1e252bc0377b8a3888c/.github/workflows/ci.yml#L83-L90

5arer commented 1 year ago

@jordanpotter you are probably right. I had issue with my vpn provider not allowing port forwarding but also switched to another docker wireguard image so cannot confirm or deny but it was definitely an issue with my vpn provider first.

jordanpotter / docker-wireguard

expose external port? #21