conflicting options: port publishing and the container type network mode

[dokbua]

I'm trying to route traffic from another container through this, using

network_mode: container:vpn

but it throws the error:

ERROR: for xxx Cannot create container for service xxx: b'conflicting options: port publishing and the container type network mode'

I also have

ports:

• 8080:8080
• 9090:9090

in the docker-compose.yml, I kinda need that, at least the 8080 as that is my admin interface to manage the containers activity.

Can you help? What am I missing?

[dperson]

The port forwarding takes place with the VPN container, not the one that reuses its IP stack.

[dokbua]

Ok, so i moved the ports to the vpn container, but now I can't connect to the ports, it times out.

So I have this service, in a docker-compose.yml:

version: "2"
services:
  myservice:
    image: linuxserver/myservice
    container_name: myservice
   ports:
     - 8080:8080
     - 9090:9090
    restart: unless-stopped

The vpn service is started in the way suggested:

docker run -it --cap-add=NET_ADMIN --device /dev/net/tun --restart=always --name vpn \
            -v /root/docker/openvpn-client/vpn:/vpn -d dperson/openvpn-client -f "" \
            -v 'somevpn.com;user;pass'

So to get myservice to route through the vpn service I need to: 1) add "network_mode: container:vpn" to the myservice container, and 2) the ports from the myservice container to the vpn contains

Like this:

version: "2"
services:
  myservice:
    image: linuxserver/myservice
    container_name: myservice
    network_mode: container:vpn
    restart: unless-stopped

docker run -it --cap-add=NET_ADMIN --device /dev/net/tun --restart=always --name vpn \
            -p 8080:8080 \
            -p 9090:9090 \
            -v /root/docker/openvpn-client/vpn:/vpn -d dperson/openvpn-client -f "" \
            -v 'somevpn.com;user;pass'

Restart both, and that should be it?

This is what I did, but when i try to go to :8080 it times out. What am I missing?

[dperson]

Check the logs for both containers (replacing <container_name> with the actual one):

docker logs <container_name>

[dokbua]

I did, but it doesn't really tell me anything I can use:

2019-09-22 10:26:20,049::INFO::[myservice:1385] Starting web-interface on 0.0.0.0:8080
2019-09-22 10:26:20,049::INFO::[_cplogging:219] [22/Sep/2019:10:26:20] ENGINE Bus STARTING
2019-09-22 10:26:20,239::INFO::[_cplogging:219] [22/Sep/2019:10:26:20] ENGINE Serving on http://0.0.0.0:8080
2019-09-22 10:26:20,240::INFO::[_cplogging:219] [22/Sep/2019:10:26:20] ENGINE Bus STARTED
2019-09-22 10:26:20,240::INFO::[myservice:1423] Starting myservice-2.3.9
2019-09-22 10:26:20,244::INFO::[dirscanner:330] Dirscanner starting up
2019-09-22 10:26:20,246::INFO::[urlgrabber:82] URLGrabber starting up
2019-09-22 10:26:20,246::INFO::[panic:209] Launching browser with http://127.0.0.1:8080/myservice
2019-09-22 10:26:20,289::INFO::[postproc:187] Completed Download Folder /config/Downloads/complete is not on FAT
2019-09-22 10:26:22,603::INFO::[__init__:1217] Could not determine system certificate validation quality due to connection problems

Sun Sep 22 19:01:23 2019 library versions: OpenSSL 1.1.1c  28 May 2019, LZO 2.10
Sun Sep 22 19:01:23 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]139.162.220.234:1194
Sun Sep 22 19:01:23 2019 UDP link local: (not bound)
Sun Sep 22 19:01:23 2019 UDP link remote: [AF_INET]139.162.220.234:1194

Can you confirm if I started the containers correctly?

[dperson]

Do you not get messages in the VPNs logs including Peer Connection Initiated, TUN/TAP device tun0 opened, and about setting an IP on tun0? If you're not, it seems that your VPN isn't connecting.

[dokbua]

No, the above is all I get.

I enter the vpn host, user and password just as I would in openvpn, where it works ok.

I'm not so sure about the certificate explanation - I've taken the certificate, including BEGIN/END lines, out of the .ovpn file, and added to a vpn-ca.crt file in the vpn directory, linked to the container.

[dperson]

The certificate can be embedded, but not if you're using the -v 'somevpn.com;user;pass', as that will overwrite the configuration file. If you want to use the config from your VPN provider, make sure to add the auth-user-pass /vpn/vpn.auth line to the VPN config file, and a /vpn/vpn.auth file with your username and password each on their own line.

[ZimboPro]

@dperson I am having a similar issue, I want to route the traffic of a Jenkins docker container

I used client tickets so used docker run -it --cap-add=NET_ADMIN --device /dev/net/tun --name vpn -p 8080:8080 -p 50000:50000 -v /path/to/vpn:/vpn -d dperson/openvpn-client

and then

docker restart vpn

and finally

docker run -it --name jenkins -v /path/to/jenkins_home:/var/jenkins_home --net=container:vpn jenkins/jenkins:lts

When trying to access port 8080 it just times out and the logs for the vpn container is

+ exec sg vpn -c 'openvpn --cd /vpn --config /vpn/vpn.conf --script-security 2 --redirect-gateway def1 ' Tue Jun 2 12:09:13 2020 OpenVPN 2.4.8 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Feb 7 2020 Tue Jun 2 12:09:13 2020 library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10 Tue Jun 2 12:09:13 2020 WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info. Tue Jun 2 12:09:13 2020 Socket Buffers: R=[212992->212992] S=[212992->212992] Tue Jun 2 12:09:13 2020 UDP link local: (not bound) Tue Jun 2 12:09:14 2020 SENT CONTROL [server]: 'PUSH_REQUEST' (status=1) Tue Jun 2 12:09:14 2020 PUSH: Received control message: 'PUSH_REPLY,route ...,push-continuation 2' Tue Jun 2 12:09:14 2020 PUSH: Received control message: 'PUSH_REPLY,route ..,push-continuation 2' Tue Jun 2 12:09:14 2020 PUSH: Received control message: 'PUSH_REPLY,route ...,push-continuation 1' Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: timers and/or timeouts modified Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: --ifconfig/up options modified Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: route options modified Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: peer-id set Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: adjusting link_mtu to 1625 Tue Jun 2 12:09:14 2020 OPTIONS IMPORT: data channel crypto options modified Tue Jun 2 12:09:14 2020 Data Channel: using negotiated cipher 'AES-256-GCM' Tue Jun 2 12:09:14 2020 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key Tue Jun 2 12:09:14 2020 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key Tue Jun 2 12:09:14 2020 TUN/TAP device tun0 opened Tue Jun 2 12:09:14 2020 TUN/TAP TX queue length set to 100 Tue Jun 2 12:09:14 2020 /sbin/ip link set dev tun0 up mtu 1500 Tue Jun 2 12:09:14 2020 /sbin/ip addr add dev tun0 local xx.xx.x.226 peer xx.xx.x.225 Tue Jun 2 12:09:14 2020 /sbin/ip route add xx.xx.xx.199/32 via xx.xx.xx.xx Tue Jun 2 12:09:14 2020 Initialization Sequence Completed

Can you please help

[lmaddox]

I'm doing the classic VPN-TOR-VPN sandwich using two instances of dperson/openvpn-client, and, on the--well, the first VPN- in the daisychain, but the second VPN instance to be started--I'm getting:

    ERROR: Cannot ioctl TUNSETIFF tun0: Resource busy (errno=16)

Is there some way to specify tun1 or some other workaround?

UPDATE: tun0 seems to be coming from the vpn configs I downloaded, not from this project. My mistake. sed -i 's/tun0/tun1/g' ... seems to have fixed it. Further testing required.

[alexpovel]

I went through almost the exact same process as @dokbua , with the same results. There's a mapping of 8080:8080 on the VPN container, and a service A that needs to run through a VPN attached to that container (network_mode: container:vpn). The port in question 8080, is published fine on the host and also found inside the VPN container:

$ root@7ac084e28006:/# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
# other crap...
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      -
# other crap...

Looks like that on the host too (8080 bound on all interfaces).

We can't do any of --add-host, --hostname, --dns, --dns-search, --dns-option, --mac-address, --publish, --publish-all, --expose (or the docker-compose equivalents) to try to hack something together, they are invalid in container:<name> mode.

curl on the service A works just fine from inside the VPN container. It seems like the routing is broken only from host to VPN container.

Note that I'm not using dperson/openvpn-client but a simple, custom Dockerfile (requires a docker secret):

FROM debian

RUN apt-get update && apt-get install --no-install-recommends --yes \
    openvpn \
    openconnect \
    ca-certificates

ENTRYPOINT [ "sh", "-c", "openconnect --verbose --passwd-on-stdin --user=$VPN_USER $VPN_URL < /run/secrets/password" ]

The issues are so similar that I thought posting here is fine, and they also indicate that the issue is not with dperson/openvpn-client per se, perhaps with Docker.

[mattdale77]

I have the same issue. I have knocked up a test with two whoami containers. As soon as I direct one to share the vpn network stack and share the port on the VPN container it times out when connecting. I used to have a very similar arrangement working very nicely so I'm not sure what has changed

[ar-jan]

I had the same issue using the wireguard image, and fixed it through this:

When routing via Wireguard from another container using the service option in docker, you might lose access to the containers webUI locally. To avoid this, exclude the docker subnet from being routed via Wireguard by modifying your wg0.conf like so (modifying the subnets as you require):

[Interface] PrivateKey = Address = 9.8.7.6/32 DNS = 8.8.8.8 PostUp = DROUTE=$(ip route | grep default | awk '{print $3}'); HOMENET=192.168.0.0/16; HOMENET2=10.0.0.0/8; HOMENET3=172.16.0.0/12; ip route add $HOMENET3 via $DROUTE;ip route add $HOMENET2 via $DROUTE; ip route add $HOMENET via $DROUTE;iptables -I OUTPUT -d $HOMENET -j ACCEPT;iptables -A OUTPUT -d $HOMENET2 -j ACCEPT; iptables -A OUTPUT -d $HOMENET3 -j ACCEPT; iptables -A OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT PreDown = HOMENET=192.168.0.0/16; HOMENET2=10.0.0.0/8; HOMENET3=172.16.0.0/12; ip route del $HOMENET3 via $DROUTE;ip route del $HOMENET2 via $DROUTE; ip route del $HOMENET via $DROUTE; iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT; iptables -D OUTPUT -d $HOMENET -j ACCEPT; iptables -D OUTPUT -d $HOMENET2 -j ACCEPT; iptables -D OUTPUT -d $HOMENET3 -j ACCEPT

I saw a similar comment on OpenVPN, but haven't tried this out:

ROUTE: '192.168.0.0/16' this value is adding a static route inside the VPN container. Provide a subnet that reflect your home LAN addressing. This is needed in order to reach containers connected to the VPN from your local network - if you misconfigure this part, containers will work, but you wont be able to access them from your laptop or PC in local network. You will still be able to access containers hidden behind VPN using host IP address and port exposed for the container as before.

[bdarley5]

I solved this by adding the required connecting container port to the VPN container and then removing them from the connecting container. Worked as expected