search

haugene / docker-transmission-openvpn

Docker container running Transmission torrent client with WebUI over an OpenVPN tunnel
GNU General Public License v3.0
4.08k stars 1.21k forks source link

Unable to connect to containers after upgrade to MacOS docker desktop v4.23 #2723

Open polleke69 opened 11 months ago

polleke69 commented 11 months ago

Is there a pinned issue for this?

Is there an existing or similar issue/discussion for this?

Is there any comment in the documentation for this?

Is this related to a provider?

Are you using the latest release?

Have you tried using the dev branch latest?

Docker run config used

"version: '3.3' services: pollekevpn: image: haugene/transmission-openvpn:dev restart: unless-stopped cap_add:

Current Behavior

All related containers start up correctly and are in working order (according to the logs). The only issue I have is that I cannot connect to the container running on http://localhost:9091 with a webbrowser (safari or Brave)

a netcat (nc -v localhost 9091) gives a "connected"

if connecting from a webbrowser the connection times out with: Safari can't open the page "localhost:9091/transmission/web/ because the server unexpectedly dropped the connection...

Expected Behavior

I would expect to see the transmission web interface.

How have you tried to solve the problem?

  1. From within the container I have checked network connectivity with a dns leak test script, which gives the expected output (DNS of VPN provider).

Log output

pollekevpn | Starting container with revision: 52d432ddca774080040627e3b6ec61fc9e6b0ac7 pollekevpn | TRANSMISSION_HOME is currently set to: /config/transmission-home pollekevpn | Creating TUN device /dev/net/tun pollekevpn | Using OpenVPN provider: NORDVPN pollekevpn | Running with VPN_CONFIG_SOURCE auto pollekevpn | Provider NORDVPN has a bundled setup script. Defaulting to internal config pollekevpn | Executing setup script for NORDVPN pollekevpn | /etc/openvpn/nordvpn/.. pollekevpn | INFO: OVPN: Checking curl installation pollekevpn | INFO: OVPN: DNS resolution ok pollekevpn | INFO: OVPN: ok, configurations download site reachable pollekevpn | INFO: OVPN: Removing existing configs in /etc/openvpn/nordvpn pollekevpn | Checking NORDPVN API responses pollekevpn | INFO: OVPN:Selecting the best server... pollekevpn | INFO: OVPN: Searching for country : RO (179) pollekevpn | INFO: OVPN: Searching for group: legacy_p2p pollekevpn | INFO: OVPN:Searching for technology: openvpn_tcp pollekevpn | INFO: OVPN: Best server : ro77.nordvpn.com, load: 7 pollekevpn | Best server : ro77.nordvpn.com pollekevpn | INFO: OVPN: Downloading config: ro77.nordvpn.com.ovpn pollekevpn | INFO: OVPN: Downloading from: https://downloads.nordcdn.com/configs/files/ovpn_tcp/servers/ro77.nordvpn.com.tcp.ovpn pollekevpn | OVPN: NORDVPN: selected: ro77.nordvpn.com, VPN_PROVIDER_HOME: /etc/openvpn/nordvpn pollekevpn | Starting OpenVPN using config ro77.nordvpn.com.ovpn pollekevpn | Modifying /etc/openvpn/nordvpn/ro77.nordvpn.com.ovpn for best behaviour in this container pollekevpn | Modification: Point auth-user-pass option to the username/password file pollekevpn | Modification: Change ca certificate path pollekevpn | Modification: Change ping options pollekevpn | Modification: Update/set resolv-retry to 15 seconds pollekevpn | Modification: Change tls-crypt keyfile path pollekevpn | Modification: Set output verbosity to 3 pollekevpn | Modification: Remap SIGUSR1 signal to SIGTERM, avoid OpenVPN restart loop pollekevpn | Modification: Updating status for config failure detection pollekevpn | Setting OpenVPN credentials... pollekevpn | adding route to local network 192.168.178.0/24 via 172.18.0.1 dev eth0 pollekevpn | 2023-09-18 16:58:39 OpenVPN 2.5.5 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 14 2022 pollekevpn | 2023-09-18 16:58:39 library versions: OpenSSL 3.0.2 15 Mar 2022, LZO 2.10 pollekevpn | 2023-09-18 16:58:39 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts pollekevpn | 2023-09-18 16:58:39 NOTE: --fast-io is disabled since we are not using UDP pollekevpn | 2023-09-18 16:58:39 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication pollekevpn | 2023-09-18 16:58:39 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication pollekevpn | 2023-09-18 16:58:39 TCP/UDP: Preserving recently used remote address: [AF_INET]86.106.137.11:443 pollekevpn | 2023-09-18 16:58:39 Socket Buffers: R=[131072->131072] S=[16384->16384] pollekevpn | 2023-09-18 16:58:39 Attempting to establish TCP connection with [AF_INET]86.106.137.11:443 [nonblock] pollekevpn | 2023-09-18 16:58:39 TCP connection established with [AF_INET]86.106.137.11:443 pollekevpn | 2023-09-18 16:58:39 TCP_CLIENT link local: (not bound) pollekevpn | 2023-09-18 16:58:39 TCP_CLIENT link remote: [AF_INET]86.106.137.11:443 pollekevpn | 2023-09-18 16:58:39 TLS: Initial packet from [AF_INET]86.106.137.11:443, sid=6484fdfd 302a6858 pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=1, O=NordVPN, CN=NordVPN CA8 pollekevpn | 2023-09-18 16:58:40 VERIFY KU OK pollekevpn | 2023-09-18 16:58:40 Validating certificate extended key usage pollekevpn | 2023-09-18 16:58:40 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication pollekevpn | 2023-09-18 16:58:40 VERIFY EKU OK pollekevpn | 2023-09-18 16:58:40 VERIFY X509NAME OK: CN=ro77.nordvpn.com pollekevpn | 2023-09-18 16:58:40 VERIFY OK: depth=0, CN=ro77.nordvpn.com pollekevpn | 2023-09-18 16:58:40 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512 pollekevpn | 2023-09-18 16:58:40 [ro77.nordvpn.com] Peer Connection Initiated with [AF_INET]86.106.137.11:443 pollekevpn | 2023-09-18 16:58:40 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 103.86.96.100,dhcp-option DNS 103.86.99.100,explicit-exit-notify,comp-lzo no,route-gateway 10.7.1.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.7.1.2 255.255.255.0,peer-id 0,cipher AES-256-CBC' pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: timers and/or timeouts modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --explicit-exit-notify can only be used with --proto udp pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: compression parms modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --ifconfig/up options modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: route options modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: route-related options modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: peer-id set pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: adjusting link_mtu to 1659 pollekevpn | 2023-09-18 16:58:40 OPTIONS IMPORT: data channel crypto options modified pollekevpn | 2023-09-18 16:58:40 Outgoing Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key pollekevpn | 2023-09-18 16:58:40 Outgoing Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication pollekevpn | 2023-09-18 16:58:40 Incoming Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key pollekevpn | 2023-09-18 16:58:40 Incoming Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication pollekevpn | 2023-09-18 16:58:40 net_route_v4_best_gw query: dst 0.0.0.0 pollekevpn | 2023-09-18 16:58:40 net_route_v4_best_gw result: via 172.18.0.1 dev eth0 pollekevpn | 2023-09-18 16:58:40 ROUTE_GATEWAY 172.18.0.1/255.255.0.0 IFACE=eth0 HWADDR=02:42:ac:12:00:02 pollekevpn | 2023-09-18 16:58:40 TUN/TAP device tun0 opened pollekevpn | 2023-09-18 16:58:40 net_iface_mtu_set: mtu 1500 for tun0 pollekevpn | 2023-09-18 16:58:40 net_iface_up: set tun0 up pollekevpn | 2023-09-18 16:58:40 net_addr_v4_add: 10.7.1.2/24 dev tun0 pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 86.106.137.11/32 via 172.18.0.1 dev [NULL] table 0 metric -1 pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 0.0.0.0/1 via 10.7.1.1 dev [NULL] table 0 metric -1 pollekevpn | 2023-09-18 16:58:40 net_route_v4_add: 128.0.0.0/1 via 10.7.1.1 dev [NULL] table 0 metric -1 pollekevpn | Up script executed with device=tun0 ifconfig_local=10.7.1.2 pollekevpn | Updating TRANSMISSION_BIND_ADDRESS_IPV4 to the ip of tun0 : 10.7.1.2 pollekevpn | pollekevpn | ------------------------------------- pollekevpn | Transmission will run as pollekevpn | ------------------------------------- pollekevpn | User name: root pollekevpn | User uid: 0 pollekevpn | User gid: 0 pollekevpn | ------------------------------------- pollekevpn | pollekevpn | Updating Transmission settings.json with values from env variables pollekevpn | Attempting to use existing settings.json for Transmission pollekevpn | Successfully used existing settings.json /config/transmission-home/settings.json pollekevpn | Overriding bind-address-ipv4 because TRANSMISSION_BIND_ADDRESS_IPV4 is set to 10.7.1.2 pollekevpn | Overriding download-dir because TRANSMISSION_DOWNLOAD_DIR is set to /data/completed pollekevpn | Overriding incomplete-dir because TRANSMISSION_INCOMPLETE_DIR is set to /data/incomplete pollekevpn | Overriding rpc-password because TRANSMISSION_RPC_PASSWORD is set to [REDACTED] pollekevpn | Overriding rpc-port because TRANSMISSION_RPC_PORT is set to 9091 pollekevpn | Overriding rpc-username because TRANSMISSION_RPC_USERNAME is set to pollekevpn | Overriding watch-dir because TRANSMISSION_WATCH_DIR is set to /data/watch pollekevpn | sed'ing True to true pollekevpn | STARTING TRANSMISSION pollekevpn | Transmission startup script complete. pollekevpn | 2023-09-18 16:58:40 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this pollekevpn | 2023-09-18 16:58:40 Initialization Sequence Completed

HW/SW Environment

- OS: MacOS Ventura 13.5.2
- Docker: Docker Desktop 4.23

Anything else?

I am not completely sure, but it looks like after upgrading to docker desktop 4.23 this issue started...

polleke69 commented 11 months ago

Just downgraded docker desktop to 4.22.1 and its working again.

pkishino commented 11 months ago

this is a docker issue.. please check after upgrade if your file sharing changed to VirtioFS instead of gRPC Fuse..if so, change back to Fuse..this causes issues and there is a long open issue for this on the docker for-mac issue tracker.. I run this container on macOS with 4.23 and it works fine using Fuse

polleke69 commented 11 months ago

I'm sorry to say, but upgrading to 4.23 again, changing VirtioFS to gRPC Fuse doesn't work...

pkishino commented 11 months ago

After upgrading, did you try to stop/start the container again? Not just start after the upgrade of docker? and you are trying to connect to http://localhost:9091/transmission/web/ right?

polleke69 commented 11 months ago

Yes, I even Rebooted and recreated the containers...

Diegus83 commented 11 months ago

I experienced the same issue today with Docker 4.23

I'm running macOS 11.7.10 so the VirtioFS option is not available for me (macOS 12 and above only.

So my settings are still set to gRPC Fuse.

I'm installing 4.22 now and will update if this changes anything.

pkishino commented 11 months ago

Strange, I created a new container on my Mac rubbing 13.6 with 4.23 docker using the 5.2.0 tag and had no issues, tested to load a few torrents etc..

Diegus83 commented 11 months ago

I installed 4.22 from Docker's website (my previous install of 4.23 was using brew) and it is working.

I didn't had to recreate my containers, simply started them and could connect to Transmission on the first try.

One thing to note is that I checked my compose file and I'm running the 4.1 image, I never got around to update past that.

So it seems whatever this issue is affects newer and older versions of the image but was introduced in Docker 4.23

Is there any other information I could provide to help?

pkishino commented 11 months ago

please check here..im also trying to find if a related issue already exists https://github.com/docker/for-mac/issues

pkishino commented 11 months ago

curious, for those of you that it does not work are you running on apple silicone?

Diegus83 commented 11 months ago

In my case I'm running on an Intel mac mini 2011 that I just updated to Big Sur using the OpenCore patcher, so before this week I was running an old version of Docker on an even older OS (High Sierra).

The only difference I noticed with my other containers that worked fine on 4.23 is that none of them use

cap_add: 
       - NET_ADMIN

Everything else in the .yml file is standard across containers: the time zone, user and group, etc.

polleke69 commented 11 months ago

Same here, the only container that doesn't work is the one with cap_add: net_admin. All others run fine...

Running on Intel macOS Ventura 13.6 and docker desktop 4.23

pkishino commented 11 months ago

please try upgrade to 4.24 and see if problem still remains

faximan commented 11 months ago

please try upgrade to 4.24 and see if problem still remains

Same problem for me on 4.24.

polleke69 commented 11 months ago

Same issue also on 4.24...

nickpainter commented 10 months ago

Seems to also impact any docker release after 4.22.1 (118664) on windows.

mrchrisster commented 10 months ago

Can't connect to the docker container anymore from my server. I'm using latest Haugen's on MacOS as well. No issues before I upgraded docker .. Is there a solution? I tried 4.23 and 4.24 - both didn't work. Downgrading to 4.22 made it work again.

pkishino commented 10 months ago

Downgrade to 4.22

On Sun, 8 Oct 2023 at 08:22, Christoph Helms @.***> wrote:

Can't connect to docker anymore. I'm using latest Haugen's on MacOS as well. No issues before I upgraded docker .. Is there a solution?

— Reply to this email directly, view it on GitHub https://github.com/haugene/docker-transmission-openvpn/issues/2723#issuecomment-1751852053, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7OFYRIDEUKDTYVTFYXMILX6HP4TAVCNFSM6AAAAAA45A4V4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHA2TEMBVGM . You are receiving this because you modified the open/close state.Message ID: @.*** com>

pkishino commented 10 months ago

That is, downgrade docker

On Sun, 8 Oct 2023 at 08:38, Patrick Kishino @.***> wrote:

Downgrade to 4.22

On Sun, 8 Oct 2023 at 08:22, Christoph Helms @.***> wrote:

Can't connect to docker anymore. I'm using latest Haugen's on MacOS as well. No issues before I upgraded docker .. Is there a solution?

— Reply to this email directly, view it on GitHub https://github.com/haugene/docker-transmission-openvpn/issues/2723#issuecomment-1751852053, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7OFYRIDEUKDTYVTFYXMILX6HP4TAVCNFSM6AAAAAA45A4V4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHA2TEMBVGM . You are receiving this because you modified the open/close state.Message ID: @.*** com>

pkishino commented 9 months ago

this should be fixed, cannot encounter this any longer

polleke69 commented 9 months ago

@pkishino Which Docker Desktop version are u using?

pkishino commented 9 months ago

4.25.0

polleke69 commented 9 months ago

Any specific settings that should be mentioned?

faximan commented 9 months ago

Still seeing this on 4.25.0, I'm staying on 4.22.1 which works well.

My compose:

transmission-openvpn:
    container_name: transmission
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    volumes:
      - '/path:/config'
      - '/path:/data'
    environment:
      - OPENVPN_PROVIDER=PIA
      - OPENVPN_CONFIG=sweden
      - OPENVPN_USERNAME=user
      - OPENVPN_PASSWORD=pass
      - LOCAL_NETWORK=192.168.2.0/24
      - GLOBAL_APPLY_PERMISSIONS=true
      - TRANSMISSION_DOWNLOAD_DIR=/data
      - TRANSMISSION_INCOMPLETE_DIR_ENABLED=false
    logging:
      driver: json-file
      options:
        max-size: 10m
    ports:
      - '9091:9091'
    image: haugene/transmission-openvpn:5.2.0
    privileged: true
mrchrisster commented 9 months ago

Thanks for your report Alex. In that case I will hold off on updating

pkishino commented 9 months ago

Sorry, my bad. They WireGuard beta version works on 4.25.0 but not the standard openvpn version strangely.. that one doesn’t work part 4.22.. haven’t had time recently to check the docker for Mac issue board in case anyone else might have a chance to

polleke69 commented 9 months ago

Have been searching for a couple of weeks now, tried this, tried that, but the only solution for now I found to "work" is to separate the openvpn and other containers but do place them in the same docker network. Not ideal, because the traffic from the other containers will not go through the vpn (transmission traffic will).

At the moment I'm also running on 4.22.1

`version: '3.3'

networks: vpn: name: vpn driver: bridge external: false

services: vpn: image: vpn:latest container_name: vpn cap_add:

ThePragmaticArt commented 8 months ago

Just revisiting this and tried to update to 4.26.1 (131620) with the same ongoing issues and no resolution. Anyone know what issue is being tracked by Docker for this?

wivaku commented 8 months ago

Worth considering in general: split VPN and the standard (Transmission) app(s).

https://code.mendhak.com/run-docker-through-vpn-container/

Support for OpenVPN as well as WireGuard. Simple to switch / set up:

version: "3"
services:
  vpn:
    image: qmcgaw/gluetun
    container_name: vpn
    # ...
    ports:
      # Transmission ports:
      - "0.0.0.0:9091:9091/tcp"   # <-- ports go here, not below
      - 51413:51413/tcp
      - 51413:51413/udp
  transmission:
    image: lscr.io/linuxserver/transmission:latest
    network_mode: "service:vpn"  # <-- important bit, don't forget
    # ...
    #ports:
    #  - "9091:9091"   # <-- ports don't go here

From browser you can continue to use http://localhost:9091. In other Docker containers that refer to Transmission use http://vpn:9091 instead of http://transmission:9091.

No Docker issues.

faximan commented 6 months ago

Just revisiting this and tried to update to 4.26.1 (131620) with the same ongoing issues and no resolution. Anyone know what issue is being tracked by Docker for this?

Bumping this. I believe I have other issues with this old version of Docker Desktop and I'd love to be able to upgrade, but this issue is blocking that.

Anybody has any new insights since last year?

jacobonorte commented 6 months ago

I second the above. I am trying to transition into MAC and I am on the latest MAC OS. It is hard to find any older versions to even installer.

Do we have another way around this?

mrchrisster commented 6 months ago

I moved to another setup with wire guard for pia and QBT. There is a python script to move all torrents to QBT. Works with latest docker. Also a great speed improvement (from 2.5 MB/s to 25MB/s)

fortinmike commented 6 months ago

@mrchrisster Do you think this alternative setup works with the latest Docker because it's using Wireguard instead of OpenVPN? Or is there something else that this image does that's triggers the Docker bug/limitation?

I'd be interested in details about your alternative setup and how it compares to docker-transmission-openvpn in terms of user-friendliness.

polleke69 commented 6 months ago

I have switched to [https://github.com/qdm12/gluetun]

Works perfectly, also with latest docker desktop versions.

mrchrisster commented 6 months ago

@mrchrisster Do you think this alternative setup works with the latest Docker because it's using Wireguard instead of OpenVPN? Or is there something else that this image does that's triggers the Docker bug/limitation?

I'd be interested in details about your alternative setup and how it compares to docker-transmission-openvpn in terms of user-friendliness.

Haugene has said it works with wire guard and newest docker, so yes, it seems to be openVPN related

In terms of user friendliness, I prefer it because of the category path management system. I'm coming from a decade of transmission usage.

The best thing is you can set the network interface to wg0 so you don't need to worry about DNS leakage.

Here's my current setup: https://github.com/thrnz/docker-wireguard-pia/issues/98#issuecomment-1914932354

Glutun is great but doesn't work for pia with wire guard

faximan commented 6 months ago

@mrchrisster Thank you for this - I have made the switch now as well and used your setup as inspiration. Works great.

With ~1300 torrents qBit is also more responsive in the UI, especially while downloading where for some reason Transmission always used to freeze up for me. Transfers also start quicker and I can reach higher max speeds (maybe that's because Wireguard instead of OpenVPN, not sure).

alexdawn commented 4 months ago

Same issue on 4.28

ssuess commented 4 months ago

same issue on 4.29

karimelc commented 2 months ago

I'm on 4.30.0 with the same problem but it works by putting the address in the ports definition like 192.168.0.1:9091:9091. Is there any downside to this?

portah commented 2 months ago

Somehow, found solution, but forgot to press post button, at #2847. It looks like bridge network from MacOS Docker Settings should be add as local network to the config.

Screenshot 2024-06-01 at 12 03 00 PM

The network from Docker's Resources settings should be added to 

       - LOCAL_NETWORK=172.16.16.0/24,192.168.65.0/24

I have not found what exactly got screwed by openvpn routing. But here is working routing for me:

root@23e5d7909fab:/# ip route
0.0.0.0/1 via 10.100.0.1 dev tun0 
default via 172.23.0.1 dev eth0 
10.100.0.0/24 dev tun0 proto kernel scope link src 10.100.0.2 
128.0.0.0/1 via 10.100.0.1 dev tun0 
169.150.232.156 via 172.23.0.1 dev eth0 
172.16.16.0/24 via 172.23.0.1 dev eth0 
172.23.0.0/16 dev eth0 proto kernel scope link src 172.23.0.2 
192.168.65.0/24 via 172.23.0.1 dev eth0

Definitely routing/bridging with new docker is a mess. On my host system (MacOS) no 192.168.65.0/24 network at all, but 192.168.64.0 on bridge100. I do not know if Docker supposed to do NAT or just routing, but at least I would think they would use original host IP like 172.16.16.11 in my case. And my nginx proxy use 172.16.16.11:9091 as transmission url. My router not aware that there is 192.168.64.0/24 somewhere on the network...

pkishino commented 2 months ago

Interesting find.. haven’t looked at the docker Mac issue board in ages.. maybe this is discussed there

mtrolley commented 1 month ago

@portah's suggestion worked for me as well, thanks!

ssuess commented 1 month ago

@portah's suggestion works for me too!! Thanks!!