Documentation: advanced port forwarding setup

[ddelange]

Is this urgent?: No

Host OS (approximate answer is fine too): Ubuntu Desktop 20.10

CPU arch or device name: aarch64

What VPN provider are you using: private internet access

What is the version of the program:

Running version latest built on 2021-05-19T17:58:32Z (commit a3751a7)

What's the problem 🤔

I've been struggling to host a docker container on the forwarded port:

• situation
  • gluetun successfully writes pia's forwarded port to the bind mounted volume
  • gluetun has ports statements disabled -- only want to use this VPN connection within this docker-compose and ideally not interfere with the rest of host's network
  • added container with network_mode: "service:gluetun", running owncloud (LAMP stack) on 8080 (not easy/recommended to change away from 8080) -- IP from this container corresponds to public VPN IP
• goal
  • i want to visit http://public_vpn_ip:forwarded_port in some browser outside my network, which should resolve to the owncloud container -- no HTTPS at all for now as it will be tricky to get letsencrypt to work for the DNS which is not accessible on port 80 as the DNS points to VPN IP
• dead ends
  • cannot use ports compose statement in combination with network_mode: "service:gluetun"
  • nginx-proxy does not work with gluetun networking setup, requires ports compose statement. also, it proxies exposed container port XXXX to 80, instead of proxying container port 80 to forwarded port XXXX
  • totally lost setting up traefik, not sure if applicable and/or compatible with gluetun
  • tried with explicitly defined networks: gluetun-network at bottom of compose file (so I could use the ports statement) and added to both services, but results in owncloud not being under the VPN IP anymore

any help reaching my owncloud container through the forwarded port would be greatly appreciated!

What are you using to run your container?: Docker Compose

version: "3.7"
services:
  gluetun:
    image: qmcgaw/gluetun:latest
    container_name: gluetun
    cap_add:
      - NET_ADMIN
    # ports:
    #   - 8888:8888/tcp # HTTP proxy
    #   - 8388:8388/tcp # Shadowsocks
    #   - 8388:8388/udp # Shadowsocks
    #   - 8000:8000/tcp # Built-in HTTP control server
    volumes:
      - ./storage/gluetun:/gluetun
      - ./storage/forwarded_port_dir:/tmp/forwarded_port_dir
    environment:
      - VPNSP=private internet access
      - PORT_FORWARDING=on
      - PORT_FORWARDING_STATUS_FILE=/tmp/forwarded_port_dir/forwarded_port
      - OPENVPN_USER=${OPENVPN_USER}
      - OPENVPN_PASSWORD=${OPENVPN_PASSWORD}
      - REGION=${REGION:-DE Berlin}
      # https://github.com/qdm12/gluetun/wiki/Environment-variables
      - DOT=off
    restart: always

  owncloud:
    image: owncloud/server:${OWNCLOUD_RELEASE_DOCKER_TAG:-10.6}
    network_mode: "service:gluetun"
    restart: always
    # could extract forwarded port from gluetun bind mount, export env var in host, and then another docker-compose up
    # but ports statement doesn't work with network_mode: "service:gluetun"
    # ports:
    #   - ${OWNCLOUD_HTTP_PORT}:8080

[qdm12]

Host ports mapping

More of an FYI than a useful point;

You need to publish the ports on gluetun, as there is only one 'network host' which is gluetun. Containers you connect to gluetun use the network stack of gluetun. So technically your owncloud container has the same container IP address as gluetun. Therefore you need to have the ports in the gluetun service definition. However this seems to be irrelevant in your situation since you care about the VPN server side port forwarded, so no need to worry about that ports section.

Reverse proxy

I would recommend you to try caddy v2 instead of nginx or traefik, it's really better having tried them all. You can use qmcgaw/caddy-scratch if you want. You should use a Caddyfile to define your proxy rules, it should be easier-than-nginx to setup. And make Caddy connected to gluetun as you did for owncloud.

Listen on the forwarded port

Adapt your Caddyfile such that Caddy listens on the forwarded port, e.g. :5858, and proxies to 127.0.0.1:8080 (your LAMP stack). The only problem is that you would need to update Caddy whenever that port forwarded port changes (see the Extras)

HTTPS

• You can use https://caddyserver.com/docs/automatic-https#dns-challenge to get https LE certs without having port 80 or 443 accessible
• You can use qmcgaw/ddns-updater connected to the gluetun container to dynamically update your domain name whenever your VPN IP address changes

That way you should be able to access https://yourdomain.com:forwarded_port 😉

Note that these may miss small things as I'm pulling this off from the top of my head, but feel free to add missing steps, I'll most likely add a page in the Wiki.

Extras

Auto-update the forwarded port in Caddy

Caddy has an HTTP API listening on port 2019 that you can enable (see the Caddyfile config options). You could have program running in gluetun periodically fetching the forwarded port with the gluetun control http server and the patching the Caddy configuration through the Caddy API. If you managed to each all the steps, I can help code that for you if you want.

Use subpaths to split to different services

You can make Caddy to use subpaths such as /owncloud and proxy to owncloud. That way you could split it to more services running in gluetun.

[ddelange]

Hi @qdm12, thanks a million for your thoughtful and structured response.

I will dive into Caddy ASAP and report back!

Also thanks for elaborating on HTTPS (and keeping me from giving up on security), sounds indeed like Caddy is the way to go for me 💪

[ddelange]

Hi again!

My progress:

• [x] DDNS
• [x] Update forwarded port in Caddyfile (http_port), push using Caddy API @ http://localhost:2019/load

  #!/usr/bin/env bash
  # auto_update_port.sh
  while true; do
  # upon gluetun startup, this file doesn't exist: skip updating
  current_port="$(cat /tmp/forwarded_port_dir/forwarded_port || echo '')"
  if grep -q "${current_port}" /caddydir/Caddyfile ; then
      :
  else
      echo "Updating Caddy"
      # update (no /g here because the port is only mentioned once)
      sed -ri "s/http_port [0-9]+/http_port ${current_port}/" /caddydir/Caddyfile
      # reload
      curl -X POST "http://localhost:2019/load" \
          -H "Content-Type: text/caddyfile" \
          --data-binary @/caddydir/Caddyfile
  fi
  sleep 10
  done

  forwarded_port_bind_mount_listener:
  build:
    context: .
    dockerfile: bash-alpine.Dockerfile
  network_mode: "service:gluetun"
  volumes:
    - ./auto_update_port.sh:/auto_update_port.sh
    - ./storage/forwarded_port_dir:/tmp/forwarded_port_dir
    - ./caddydir:/caddydir
  command: ["bash", "/auto_update_port.sh"]
  restart: always

• [x] all containers specify network_mode: "service:gluetun"
• [ ] access http://yourdomain.com:forwarded_port
• [ ] https LE certs without having port 80 or 443 accessible
• [ ] access https://yourdomain.com:forwarded_port
• [ ] subpaths to split to different services

So making my way down the TODO list, I've reached a bit of a catch 22:

• With an owncloud (80) container and a Caddyfile

  {
    http_port 42424
    admin 127.0.0.1:2019
  }
  
  reverse_proxy 127.0.0.1:80

  • caddy starts and runs (with some tls warnings, ignoring for now)
  • owncloud startup (takes some time) errors '80 already in use'

• Switching to an ocis (9250 by default) container and a Caddyfile

  {
    http_port 42424
    admin 127.0.0.1:2019
  }
  
  reverse_proxy 127.0.0.1:9250

  • either ocis, or caddy starts correctly, the other will exit (depending who did the bind mount first I guess)
  • caddy is the more verbose of the two: run: loading initial config: loading new config: http app module: start: tcp: listening on :9250: listen tcp :9250: bind: address already in use

Trying to keep compose as basic as possible for testing purposes:

services:
  # gluetun unchanged

  # ddns added

  # forwarded_port_bind_mount_listener added, updates/reloads caddy when needed

  caddy:
    image: qmcgaw/caddy-scratch
    container_name: caddy
    volumes:
     - ./caddydir:/caddydir
    network_mode: "service:gluetun"
    restart: always

  ocis:
    image: "owncloud/ocis:${OCIS_DOCKER_TAG:-latest}"
    container_name: ocis
    network_mode: "service:gluetun"
    environment:
      - OCIS_RUNTIME_PORT=9250

My search for similar issues left me stranded, this post is the closest I got. Any more wisdom you can spare? :)

[ddelange]

Next (failed) try which confirms caddy occupying 9250:

• Simplify Caddyfile

  :42424
  
  reverse_proxy ocis_host:9250

• Add env var OCIS_RUNTIME_HOST=ocis_host to the ocis service in docker-compose
• Start up whole stack except for ocis
• Start up ocis -> exit 1 runtime address already in use

[ddelange]

And probably a separate issue with PIA PF, there seems no open connection over the port into the docker network although gluetun reports PF success on startup:

• Simplify Caddyfile

  :28741
  
  respond "Hello, world!"

• Verify caddy reachable within gluetun network from within ocis pod:

  # curl localhost:28741
  Hello, world!

• From within same ocis pod, to gluetun's reported vpn_ip/ddns_ip, connection won't establish as suggested in docs linked below:

  # gluetun reports success:
  # INFO firewall: setting allowed input port 28741 through interface tun0..
  $ curl *vpn_ip*:28741
  connect to *vpn_ip* port 28741 failed: Operation timed out

• But from another machine [docs], it is also not reachable: a ERR_CONNECTION_REFUSED in the browser as well as from curl

[qdm12]

Sorry I'm still reading / processing all of that, thanks for sharing !

From within same ocis pod, to gluetun's reported vpn_ip/ddns_ip, connection won't establish as suggested in docs linked below

PIA has been notoriously impossible to access its forwarded port. We and other users did a lot of experiments on another issue, and it's not gluetun's fault but rather PIA, as it doesn't even work with plain openvpn with their scripts. HOWEVER it works with the yougetsignal port checker website and for torrenting port forwarding. The only explanation I have so far is they do deep packet inspection on the port forwarded traffic and drop traffic they don't like. Their support wasn't helpful but you can try ask them.

Alternatively go with another provider like Mullvad/VyprVPN that also offers port forwarding.

EDIT: See this comment : https://github.com/qdm12/gluetun/issues/53#issuecomment-634329721

[ddelange]

PIA replied that their service does not support incoming connections over a forwarded port. With that, I'll close this issue. Thanks again for all the help!

[qdm12]

Oh what that's so strange... what's the point then 😄 I'll document this in the wiki

[ddelange]

I also don't understand the answer (I was asking specifically about hosting a webserver on the forwarded port), because incoming connections on the forwarded port seem to work fine e.g. for P2P protocols

[qdm12]

Added your comments and linked to this issue at https://github.com/qdm12/gluetun/wiki/Private-internet-access#warning

I had the same experience back when I was a PIA subscriber as well.

[prabbit237]

I haven't actually tried it with gluetun yet but port-forwarding with PIA using https://github.com/pia-foss/manual-connections and the openvpn protocol works fine. I found this thread when troubleshooting some issues with the port forwarding

What seems to not work properly (at least with pia-foss/manual-connections) is port-forwarding with wireguard. I can get that to work in Windows with the native PIA GUI using wireguard and running an Apache server on the forwarded port. But trying to use wireguard and manual-connections on a Raspberry Pi failed. Switching to openvpn was fine.

sudo VPN_PROTOCOL=wireguard DISABLE_IPV6=yes DIP_TOKEN=no AUTOCONNECT=true PIA_PF=true PIA_DNS=true PIA_USER=p1234567 PIA_PASS=XXXXXX MAX_LATENCY=0.1 ./run_setup.sh

fails.

sudo VPN_PROTOCOL=openvpn DISABLE_IPV6=yes DIP_TOKEN=no AUTOCONNECT=true PIA_PF=true PIA_DNS=true PIA_USER=p1234567 PIA_PASS=XXXXXX MAX_LATENCY=0.1 ./run_setup.sh

works fine. I got an IP of 1.2.3.4 and a port 12345 and was able to then run:

ssh 1.2.3.4 -p 12345

with no issue (changing sshd to listen on the port and using the real IP & port, of course.)

[ddelange]

That's great news, time to try again:) Thanks for reporting!

[Seros]

Hey @ddelange, just out of curiosity, were you able to confirm that port forwarding with gluetun is now working as expected?

[ddelange]

hi :wave: didn't have the time yet