qdm12 / gluetun

VPN client in a thin Docker container for multiple VPN providers, written in Go, and using OpenVPN or Wireguard, DNS over TLS, with a few proxy servers built-in.
https://hub.docker.com/r/qmcgaw/gluetun
MIT License
7.69k stars 359 forks source link

Help: Very slow download speeds compared to other containers #802

Closed ispiropoulos closed 2 years ago

ispiropoulos commented 2 years ago

Is this urgent?

Yes

Host OS

Ubuntu 20.04

CPU arch

x86_64

VPN service provider

Private Internet Access

What are you using to run the container

docker-compose

What is the version of Gluetun

Running version latest built on 2021-12-14T18:21:24.225Z (commit ca82fcb)

What's the problem πŸ€”

The download speeds are very slow via gluetun. I have made a comparison with ghcr.io/wfg/openvpn-client and the results are night and day.

Testing with PIA on gluetun:

Screenshot 2022-01-19 at 11 25 28 AM

Testing with PIA on openvpn-client:

Screenshot 2022-01-19 at 11 27 50 AM

The performed tests are with PIA_ENCRYPTION=normal on gluetun and the open vpn configuration for normal mode on openvpn-client.

I have also tested with port worwarding disabled. No change.

Share your logs

john@ubuntu-server:~/docker-containers/vpn-stack$ docker logs gluetun
========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❀️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2021-12-14T18:21:24.225Z (commit ca82fcb)

πŸ”§ Need help? https://github.com/qdm12/gluetun/discussions/new
πŸ› Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
β˜• Discussion? https://github.com/qdm12/gluetun/discussions/new
πŸ’» Email? quentin.mcgaw@gmail.com
πŸ’° Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2022/01/19 11:43:31 INFO storage: merging by most recent 11120 hardcoded servers and 11120 servers read from /gluetun/servers.json
2022/01/19 11:43:31 INFO Alpine version: 3.15.0
2022/01/19 11:43:31 INFO OpenVPN 2.4 version: 2.4.11
2022/01/19 11:43:31 INFO OpenVPN 2.5 version: 2.5.4
2022/01/19 11:43:31 INFO Unbound version: 1.13.2
2022/01/19 11:43:31 INFO IPtables version: v1.8.7
2022/01/19 11:43:31 INFO Settings summary below:
|--VPN:
   |--Type: openvpn
   |--OpenVPN:
      |--Version: 2.5
      |--Verbosity level: 1
      |--Network interface: tun0
      |--Run as root: enabled
      |--Encryption preset: normal
   |--Private Internet Access settings:
      |--Regions: greece
      |--OpenVPN selection:
         |--Protocol: udp
         |--PIA encryption preset: normal
      |--Port forwarding:
         |--File path: /gluetun/forwarded_port
|--DNS:
   |--Plaintext address: 1.1.1.1
   |--DNS over TLS:
      |--Unbound:
          |--DNS over TLS providers:
              |--Cloudflare
          |--Listening port: 53
          |--Access control:
              |--Allowed:
                  |--0.0.0.0/0
                  |--::/0
          |--Caching: enabled
          |--IPv4 resolution: enabled
          |--IPv6 resolution: disabled
          |--Verbosity level: 1/5
          |--Verbosity details level: 0/4
          |--Validation log level: 0/2
          |--Username: 
      |--Blacklist:
         |--Blocked categories: malicious
         |--Additional IP networks blocked: 13
      |--Update: every 24h0m0s
|--Firewall:
   |--Outbound subnets: 192.168.1.0/24
|--Log:
   |--Level: INFO
|--System:
   |--Process user ID: 1000
   |--Process group ID: 1000
   |--Timezone: europe/athens
|--HTTP proxy:
   |--Port: 8888
|--Health:
   |--Server address: 127.0.0.1:9999
   |--Address to ping: github.com
   |--VPN:
      |--Initial duration: 6s
      |--Addition duration: 5s
|--HTTP control server:
   |--Listening port: 8000
   |--Logging: enabled
|--Public IP getter:
   |--Fetch period: 12h0m0s
   |--IP file: /tmp/gluetun/ip
|--Github version information: enabled
2022/01/19 11:43:31 INFO routing: default route found: interface eth0, gateway 172.29.0.1
2022/01/19 11:43:31 INFO routing: local ethernet link found: eth0
2022/01/19 11:43:31 INFO routing: local ipnet found: 172.29.0.0/16
2022/01/19 11:43:31 INFO routing: default route found: interface eth0, gateway 172.29.0.1
2022/01/19 11:43:31 INFO routing: adding route for 0.0.0.0/0
2022/01/19 11:43:31 INFO firewall: firewall disabled, only updating allowed subnets internal list
2022/01/19 11:43:31 INFO routing: default route found: interface eth0, gateway 172.29.0.1
2022/01/19 11:43:31 INFO routing: adding route for 192.168.1.0/24
2022/01/19 11:43:31 INFO TUN device is not available: open /dev/net/tun: no such file or directory; creating it...
2022/01/19 11:43:31 INFO firewall: enabling...
2022/01/19 11:43:31 INFO firewall: enabled successfully
2022/01/19 11:43:31 INFO healthcheck: listening on 127.0.0.1:9999
2022/01/19 11:43:31 INFO firewall: setting VPN connection through firewall...
2022/01/19 11:43:31 INFO dns over tls: using plaintext DNS at address 1.1.1.1
2022/01/19 11:43:31 INFO http proxy: listening on :8888
2022/01/19 11:43:31 INFO http server: listening on :8000
2022/01/19 11:43:31 INFO openvpn: OpenVPN 2.5.4 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Nov 15 2021
2022/01/19 11:43:31 INFO openvpn: library versions: OpenSSL 1.1.1l  24 Aug 2021, LZO 2.10
2022/01/19 11:43:31 INFO openvpn: CRL: loaded 1 CRLs from file -----BEGIN X509 CRL-----
2022/01/19 11:43:31 INFO openvpn: MIICWDCCAUAwDQYJKoZIhvcNAQENBQAwgegxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJDQTETMBEGA1UEBxMKTG9zQW5nZWxlczEgMB4GA1UEChMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxIDAeBgNVBAsTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAwHgYDVQQDExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UEKRMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxLzAtBgkqhkiG9w0BCQEWIHNlY3VyZUBwcml2YXRlaW50ZXJuZXRhY2Nlc3MuY29tFw0xNjA3MDgxOTAwNDZaFw0zNjA3MDMxOTAwNDZaMCYwEQIBARcMMTYwNzA4MTkwMDQ2MBECAQYXDDE2MDcwODE5MDA0NjANBgkqhkiG9w0BAQ0FAAOCAQEAQZo9X97ci8EcPYu/uK2HB152OZbeZCINmYyluLDOdcSvg6B5jI+ffKN3laDvczsG6CxmY3jNyc79XVpEYUnq4rT3FfveW1+Ralf+Vf38HdpwB8EWB4hZlQ205+21CALLvZvR8HcPxC9KEnev1mU46wkTiov0EKc+EdRxkj5yMgv0V2Reze7AP+NQ9ykvDScH4eYCsmufNpIjBLhpLE2cuZZXBLcPhuRzVoU3l7A9lvzG9mjA5YijHJGHNjlWFqyrn1CfYS6koa4TGEPngBoAziWRbDGdhEgJABHrpoaFYaL61zqyMR6jC0K2ps9qyZAN74LEBedEfK7tBOzWMwr58A==
2022/01/19 11:43:31 INFO openvpn: -----END X509 CRL-----
2022/01/19 11:43:31 INFO openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]154.57.3.97:1198
2022/01/19 11:43:31 INFO openvpn: UDP link local: (not bound)
2022/01/19 11:43:31 INFO openvpn: UDP link remote: [AF_INET]154.57.3.97:1198
2022/01/19 11:43:32 WARN openvpn: 'link-mtu' is used inconsistently, local='link-mtu 1557', remote='link-mtu 1542'
2022/01/19 11:43:32 WARN openvpn: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo'
2022/01/19 11:43:32 INFO openvpn: [athens402] Peer Connection Initiated with [AF_INET]154.57.3.97:1198
2022/01/19 11:43:32 INFO openvpn: TUN/TAP device tun0 opened
2022/01/19 11:43:32 INFO openvpn: /sbin/ip link set dev tun0 up mtu 1500
2022/01/19 11:43:32 INFO openvpn: /sbin/ip link set dev tun0 up
2022/01/19 11:43:32 INFO openvpn: /sbin/ip addr add dev tun0 10.6.112.5/24
2022/01/19 11:43:32 INFO openvpn: Initialization Sequence Completed
2022/01/19 11:43:32 INFO dns over tls: downloading DNS over TLS cryptographic files
2022/01/19 11:43:32 INFO healthcheck: healthy!
2022/01/19 11:43:34 INFO dns over tls: downloading hostnames and IP block lists
2022/01/19 11:43:49 WARN dns over tls: context deadline exceeded (Client.Timeout or context cancellation while reading body)
2022/01/19 11:43:49 INFO dns over tls: init module 0: validator
2022/01/19 11:43:49 INFO dns over tls: init module 1: iterator
2022/01/19 11:43:49 INFO dns over tls: start of service (unbound 1.13.2).
2022/01/19 11:43:49 INFO dns over tls: generate keytag query _ta-4a5c-4f66. NULL IN
2022/01/19 11:43:49 INFO dns over tls: generate keytag query _ta-4a5c-4f66. NULL IN
2022/01/19 11:43:50 INFO dns over tls: ready
2022/01/19 11:43:51 INFO vpn: You are running 22 commits behind the most recent latest
2022/01/19 11:43:51 INFO vpn: VPN gateway IP address: 10.6.112.1
2022/01/19 11:43:51 INFO port forwarding: Found saved forwarded port data for port 58029
2022/01/19 11:43:51 INFO port forwarding: Forwarded port data expires in 50 days
2022/01/19 11:43:51 INFO port forwarding: Port forwarded data expires in 50 days
2022/01/19 11:43:51 INFO port forwarding: port forwarded is 58029
2022/01/19 11:43:51 INFO firewall: setting allowed input port 58029 through interface tun0...
2022/01/19 11:43:51 INFO port forwarding: writing port file /gluetun/forwarded_port
2022/01/19 11:43:52 INFO ip getter: Public IP address is 154.57.3.97 (Greece, Attica, Athens)

Share your configuration

version: "3"
services:
  gluetun:
    image: qmcgaw/gluetun:latest
    container_name: gluetun
    cap_add:
      - NET_ADMIN
    environment:
      - VPNSP=private internet access
      - OPENVPN_USER=*********
      - OPENVPN_PASSWORD==*********
      - REGION=Greece
      - PIA_ENCRYPTION=normal
      - PORT_FORWARDING=on
      - HTTPPROXY=on
      - TZ=Europe/Athens
      - FIREWALL_OUTBOUND_SUBNETS=192.168.1.0/24
      - PORT_FORWARDING_STATUS_FILE=/gluetun/forwarded_port
    ports:
      - 9091:9091 # Transmission
      - 8020:8020 # pyLoad
      - 6881:6881 # Transmission
      - 8040:8000 # Gluetun HTTP Control Server
      - 8888:8888 # HTTP Proxy
      - 8766:80   # Speed Test VPN
    volumes:
      - ./gluetun-data:/gluetun
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=1
    restart: unless-stopped
  torrent:
    container_name: transmission
    image: ghcr.io/linuxserver/transmission
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Athens
    volumes:
      - ../transmission/transmission-config:/config
      - ../transmission/downloads:/downloads
      - ../transmission/watch:/watch
    network_mode: service:gluetun
    restart: unless-stopped
  downloader:
    image: lscr.io/linuxserver/pyload
    container_name: pyload
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Athens
    volumes:
      - ../pyload/pyload-config:/config
      - ../pyload/downloads:/downloads
    network_mode: service:gluetun
    restart: unless-stopped
  speedtest:
    container_name: speedtest-vpn
    image: henrywhitaker3/speedtest-tracker
    volumes:
      - ../speedtest-vpn/speedtest-config:/config
    environment:
      - TZ=Europe/Athens
      - PGID=1000
      - PUID=1000
      - OOKLA_EULA_GDPR=true
    logging:
      driver: "json-file"
      options:
        max-file: "10"
        max-size: "200k"
    network_mode: service:gluetun
    restart: unless-stopped
qdm12 commented 2 years ago

What device are you using? Which country are you physically located in?

Try disabling the firewall with FIREWALL: off maybe that puts too much load on your device?

You might want to compare with a VPN server near your location to ensure you compare processing and not some random bandwidth bottleneck.

qdm12 commented 2 years ago

There is also a memory issue #765 where gluetun uses 500mb of ram (due to servers data being kept in memory) so maybe your device is low on ram. I'm working on fixing it currently.

ispiropoulos commented 2 years ago

What device are you using? Which country are you physically located in?

Try disabling the firewall with FIREWALL: off maybe that puts too much load on your device?

You might want to compare with a VPN server near your location to ensure you compare processing and not some random bandwidth bottleneck.

First of all, congratulations for gluetun, seems like you have put a lot of effort to code / maintain.

The server is indeed in my Country (Greece), both the PIA server and the speediest server.

I just tried with environment variable - FIREWALL=off but unfortunately the same outcome.

There is also a memory issue #765 where gluetun uses 500mb of ram (due to servers data being kept in memory) so maybe your device is low on ram. I'm working on fixing it currently.

The memory and cpu usage does not seem to be high, > docker stats gluetun returned this while running the speedtest:

Screenshot 2022-01-19 at 2 38 52 PM

Weird thing; I also tried docker-wireguard-pia and linuxserver/wireguard and have similar low speeds. Only docker-openvpn-client seems to be working, but I'd love to see gluetun work for me for the convenience of port forwarding etc.

Could it be some docker image setting that makes it so slow? Unfortunately, while I am a software engineer, my experience with docker is 0.

qdm12 commented 2 years ago

First of all, congratulations for gluetun, seems like you have put a lot of effort to code / maintain.

Thanks πŸ’―

The server is indeed in my Country (Greece), both the PIA server and the speediest server.

Awesome πŸŽ–οΈ

I just tried with environment variable - FIREWALL=off but unfortunately the same outcome.

Got it. docker-openvpn-client also uses some iptables firewall so similar to gluetun. Thanks for confirming it's not that.

Weird thing; I also tried docker-wireguard-pia and linuxserver/wireguard and have similar low speeds.

With all the praises and prayers to wireguard, this is just hilarious. Anyway, this is bugging me as much as it is for you πŸ˜„

Could it be some docker image setting that makes it so slow? Unfortunately, while I am a software engineer, my experience with docker is 0.

No worry, happy to help! I checked their Dockerfile and it looks like it's just OpenVPN on alpine:3.15 like gluetun is. So probably not a docker image issue, it's the same package. Although gluetun installs both openvpn 2.4 and 2.5 to be retro-compatible, maybe that's a problem.

  1. I built and pushed gluetun with openvpn 2.5 only (not 2.4) at qmcgaw/gluetun:802 can you try? It probably won't help but who knows.
  2. When running the test with docker-openvpn-client, did you use it in the same docker-compose.yml (with transmission connected to it)?
  3. Maybe you got lucky and went on a low usage VPN server when you tested docker-openvpn-client? Did you benchmark it multiple time, as in shutting the VPN container down and re-connecting?

I also checked docker-openvpn-client's entrypoint and there is nothing extra compared to gluetun (no picking of best server etc.)

ispiropoulos commented 2 years ago

Could it be some docker image setting that makes it so slow? Unfortunately, while I am a software engineer, my experience with docker is 0.

No worry, happy to help! I checked their Dockerfile and it looks like it's just OpenVPN on alpine:3.15 like gluetun is. So probably not a docker image issue, it's the same package. Although gluetun installs both openvpn 2.4 and 2.5 to be retro-compatible, maybe that's a problem.

  1. I built and pushed gluetun with openvpn 2.5 only (not 2.4) at qmcgaw/gluetun:802 can you try? It probably won't help but who knows.

Thanks. Just tried and I’m getting this error:


========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❀️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version unknown built on an unknown date (commit unknown)

πŸ”§ Need help? https://github.com/qdm12/gluetun/discussions/new
πŸ› Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
β˜• Discussion? https://github.com/qdm12/gluetun/discussions/new
πŸ’» Email? quentin.mcgaw@gmail.com
πŸ’° Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2022/01/20 07:16:12 INFO storage: merging by most recent 11100 hardcoded servers and 11100 servers read from /gluetun/servers.json
2022/01/20 07:16:12 INFO Alpine version: 3.15.0
2022/01/20 07:16:12 ERROR exec: "openvpn2.4": executable file not found in $PATH
2022/01/20 07:16:12 INFO Shutdown successful
  1. When running the test with docker-openvpn-client, did you use it in the same docker-compose.yml (with transmission connected to it)?

Yes. Transmission, pyLoad, and that speedtest-tracker container working great with docker-openvpn-client.

  1. Maybe you got lucky and went on a low usage VPN server when you tested docker-openvpn-client? Did you benchmark it multiple time, as in shutting the VPN container down and re-connecting?

I also checked docker-openvpn-client's entrypoint and there is nothing extra compared to gluetun (no picking of best server etc.)

I think it’s not luck, also tried with other countries like France and Austria, speed difference was there as well. One thing I noticed with France is that when I connected with gluetun and run the speedtest image I showed on my initial issue post, speedtest-cli showed me a different ISP (something in Chile) although the gluetun logs showed that I was connected to France and the ip it gave me was in France. I can try again in an hour and provide logs and screenshots.

Also, I think it’s noteworthy that:

ispiropoulos commented 2 years ago

I completely removed the gluetun container, all the cached images my docker instance was not using, all the unused networks, and fetched again from gluetun:latest. And now I get fast speeds.

This is very very weird.

The weirdest part is that I also tried running the container on a raspberry pi 3b+ I have (on the same network as my main home server) just before I did that clean up and I had the same issue.

UPDATE:

After a reboot on the host machine, same problem happens again.

ispiropoulos commented 2 years ago

Update:

After that reboot I mentioned earlier, I had very poor speeds even with docker-openvpn-client.

I then tried to use the configuration files with gluetun in custom provider mode. Same results (and I have to change the host name on the conf file into an ip, hostname resolved in multiple ips, so I tried one by one with various results for each one, but generally slow speeds (1,5 - 6 mbps)

Then, I changed the protocol to tcp with the - OPENVPN_PROTOCOL=tcp environment variable and scheduled a speedtest to occur every 15 minutes overnight. With the exception of two consecutive runs, I’ve been getting way better speeds, around 40-70mbps.

qdm12 commented 2 years ago

Maybe it's just that their UDP servers are overloaded (since it's the default for Openvpn) perhaps?

Protocol wise, since you are wrapping mostly TCP traffic in UDP traffic when using UDP, it's not a matter of unreliable connection. If you get bad speeds with Wireguard as well which uses UDP, that's also strange. Maybe only OpenVPN TCP servers are not overloaded. It might be worth asking them directly πŸ€”

I'll leave the issue opened for a few more days if you find more things, and then I'll document that in the wiki page as a tip πŸ˜‰

ispiropoulos commented 2 years ago

Hi Quentin, I have not forgotten about this "issue", just being swamped with work. Been running smoothly via TCP for days. Have tried UDP again but with the same issues. I will send over an email to PIA and let you know.

RW21 commented 2 years ago

I am also facing the issue. Tried OPENVPN_PROTOCOL=tcp, but no luck. Doesn't seem like a tcp/udp problem for me...

qdm12 commented 2 years ago

It might be depending on VPN server load. I think they (PIA) have an API to fetch the load of a server if I'm not mistaken. Other than that, I can't really help as it can be due to a bunch of external factors (memory available, speed test server load, vpn server load, connection bandwidth, connection reliability, cpu load on your machine, to name a few)

RW21 commented 2 years ago

Hmm, maybe perhaps its a pia server issue. I am getting fast uploads, but slow downloads. I have tested older version of gluetun which used to work fine, but still get the same issue.

RW21 commented 2 years ago

Just a question: does - OPENVPN_PROTOCOL=tcp work when - VPNSP=private internet access Do I need to use openvpn mode to enable it?

qdm12 commented 2 years ago

@RW21 It is openvpn by default, and wireguard isn't supported for PIA yet. Anyway if you're in doubt, you can just try the settings and gluetun will complain if a setting is not valid, there is a decent validation system in place.

RW21 commented 2 years ago

:+1: Thanks for the prompt reply. For me, changing the protocol to TCP didn't effect the speed. I will try to recreate the environment in another machine.

RW21 commented 2 years ago

btw, I'm getting way better speeds on other machines. Seems to be a problem with my rasberry pi.

qdm12 commented 2 years ago

@RW21 interesting. You can check if htop if openvpn uses a lot of CPU or memory;

If it does use a lot of CPU, you can subscribe to #612 to know when wireguard is implemented for PIA, it might help reduce CPU load.

Otherwise, it might be because raspberry Pis (except 4 I think) have 100mbps ethernet so limited to 12.5MB/s, and that's most likely worst in Wifi, maybe that's the reason?

RW21 commented 2 years ago

Yes I did notice that my rasberry pi was consistently using 80% of cpu resource (not only from gluetun, but from a bunch of different services), but I'm sure this is not the root cause, since it used to work fine under this condition.

As an update, I finally upgraded from Ubuntu 21.04 to 21.10, and this somehow fixed it. But it is weird considering before the upgrade, internet speeds where fine except for gluetun. I suppose some kernel update fixed the problem.

Anyways, thanks for your help! Wireguard on PIA looks very promising. I am looking forwards for it :+1:

qdm12 commented 2 years ago

Cool! I'll close this issue for now then.

Also regarding Wireguard, don't expect too much from it. It does bring benefits if the VPN server is nearby and you CPU is single-thread bad (:eyes: raspberry pi), but otherwise it will likely give the same performance as openVPN.