search

haugene / docker-transmission-openvpn

Docker container running Transmission torrent client with WebUI over an OpenVPN tunnel
GNU General Public License v3.0
4.14k stars 1.21k forks source link

Update PIA configs to next gen servers #1334

Closed haugene closed 4 years ago

haugene commented 4 years ago

Which provider? PIA

Where are the configs? Should probably update to the next gen servers.

https://www.privateinternetaccess.com/helpdesk/news/posts/august-19th-2020-important-updated-server-changes-and-related-issues

biggeeus commented 4 years ago

@Biggus-Geeus @jscoys @GabrielJean Just merged #1413 that should fix the issue of torrents not starting when specifying TRANSMISSION_PEER_LIMIT_GLOBAL, TRANSMISSION_MAX_PEERS_GLOBAL, etc. I'm just assuming this also solves your problem @jscoys. Please provide your docker-compose/docker run command and logs if it doesn't.

@haugene tested and now all of the environment variables I am passing in my docker compose pass the correct values, thanks :-)

jscoys commented 4 years ago

@haugene hey thx a lot man, it indeed solved the issue and everything is working well. Thx again for your good job! 🎉

Coldorak commented 4 years ago

Hi all. I seem to be having the same issue as @mrdink - #1334 (comment) - essentially the container starts up, stays up, but also doesn't initiate the OpenVPN profile. Using debug as suggested by @haugene , log shows that it gets stuck in an endless loop trying to "modify configs for this container" - extract: haugene_transmissionopenvpn_dev_log.txt. Any ideas appreciated.

I got the same problem, and in my ovpn file I quickly get something like this: auth-user-pass /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt

stephane-j commented 4 years ago

Just wanted to say that I changed from latest to dev tag and it worked first try! Good job!

superkrups20056 commented 4 years ago

Anything else keeping this build from being merged to stable? Looks like all the major bugs are ironed out.

biggeeus commented 4 years ago

Tinyproxy isn't working for me on the dev build. I get the following warning in the log.

STARTING TINYPROXY Found config file /etc/tinyproxy/tinyproxy.conf, updating settings. Setting tinyproxy port to 8888

Kirkerino commented 4 years ago

Hi all. I seem to be having the same issue as @mrdink - #1334 (comment) - essentially the container starts up, stays up, but also doesn't initiate the OpenVPN profile. Using debug as suggested by @haugene , log shows that it gets stuck in an endless loop trying to "modify configs for this container" - extract: haugene_transmissionopenvpn_dev_log.txt. Any ideas appreciated.

I got the same problem, and in my ovpn file I quickly get something like this: auth-user-pass /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt

Same issue here! Will probably try starting fresh later today to see if there's something which isn't playing nice when changing from latest to dev.

Nighthawk3D commented 4 years ago

Hi all. I seem to be having the same issue as @mrdink - #1334 (comment) - essentially the container starts up, stays up, but also doesn't initiate the OpenVPN profile. Using debug as suggested by @haugene , log shows that it gets stuck in an endless loop trying to "modify configs for this container" - extract: haugene_transmissionopenvpn_dev_log.txt. Any ideas appreciated.

I got the same problem, and in my ovpn file I quickly get something like this: auth-user-pass /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt /config/openvpn-credentials.txt

Same issue here! Will probably try starting fresh later today to see if there's something which isn't playing nice when changing from latest to dev.

Getting the same issue as well. Attaching docker-compose .yaml

By chance are you on a Synology? docker-compose.txt

superkrups20056 commented 4 years ago

I'm on synology. Looks like dev build has taken a step backwards for us.

I have three problems:

Log with debug environmental variable enabled:

Transmission-OpenVPN.txt (change the file type to html before viewing this)

Docker compose:

Transmission 2.txt

biggeeus commented 4 years ago

I have found the issue with tinyproxy not working

In the file /opt/tinyproxy/start.sh

at the end of the file there is the line /etc/init.d/tinyproxy start

this need to be changed to cd /etc/init.d tinyproxy start

Not sure why this fixes it but it works for me.

Kirkerino commented 4 years ago

Getting the same issue as well. Attaching docker-compose .yaml

By chance are you on a Synology? docker-compose.txt

I'm on Unraid! (6.8.3)

haugene commented 4 years ago

Weird stuff with the endless loop of "modifying configs" stuff. If it's a platform issue that might make it easier to make a broad fix, but need to narrow down what's causing it :thinking:

I'll have a look at the tinyproxy setup now. But since it seems there are barely any legacy PIA servers left I'm thinking of merging the changes to master branch soon.

haugene commented 4 years ago

@Kirkerino @phasma343 I just pushed some minor changes, can you test it? I'm not super hopeful but I did see some output that I didn't like and I think running the modification script with xargs is a better approach.

@superkrups20056 Are you sure you have pulled the latest version of the dev image?

Nighthawk3D commented 4 years ago

@Kirkerino @phasma343 I just pushed some minor changes, can you test it? I'm not super hopeful but I did see some output that I didn't like and I think running the modification script with xargs is a better approach.

@haugene No more loop during configuration and health check comes back good. The latest changes seem to have done the trick.

Attached the container logs for you for validation that everything looks clean. If you need me to enable debug logging and re-create the container I can do that for you as well.

docker_transmission_dev_log.txt

Kirkerino commented 4 years ago

@Kirkerino @phasma343 I just pushed some minor changes, can you test it? I'm not super hopeful but I did see some output that I didn't like and I think running the modification script with xargs is a better approach.

@superkrups20056 Are you sure you have pulled the latest version of the dev image?

Oh, that actually fixed it! It's working perfectly now, thank you very much!

haugene commented 4 years ago

@Kirkerino @phasma343 Excellent! :tada: Thanks for your quick feedback. Really glad to be able to close that. Still don't know exactly why, but I'll leave it alone for now :sweat_smile: Should have known, xargs is always the right choice. I just thing find -exec looks smoother. Never again ;)

@Biggus-Geeus I hope I just pushed a fix for the tinyproxy startup. It seems that tinyproxy actually installs to /usr/bin/tinyproxy as well so I just call it from the path if it's there. Doing it only if it's found for now, need to check how this is in the arm-based images. Hopefully we can use that start method for all images. Test it and let me know :+1:

justin-peacock commented 4 years ago

Huzzah! I pulled the latest dev and it's working for me on Synology. This was a fresh setup so I'm not sure if my issues were with trying to use old config.

haugene commented 4 years ago

FYI all. I just pushed all the changes to master :boom: :rocket:

I think we've ironed out most of the issues and now let's hope it proves to be relatively stable going forward. There are still stability issues or at least possible improvements to the PIA port forwarding script etc. But this thread has done it's job soon I think. It's time to close it and open new ones for any bugs still left.

Thank you all for helping out with the testing and debugging :pray:

biggeeus commented 4 years ago

@Biggus-Geeus I hope I just pushed a fix for the tinyproxy startup. It seems that tinyproxy actually installs to /usr/bin/tinyproxy as well so I just call it from the path if it's there. Doing it only if it's found for now, need to check how this is in the arm-based images. Hopefully we can use that start method for all images. Test it and let me know 👍

Works perfectly, thank you

11jwolfe2 commented 4 years ago

A newbie here but this doesn't seem to have fixed the issue on arm. I am pulling latest-armhf, as instructed but it doesn't seem to have fixed the issue. am I missing something or is this a feature the raspberry pi community will have to wait for.

JB09 commented 4 years ago

I am pulling latest-armhf, as instructed but it doesn't seem to have fixed the issue.

I am using the image from ~12 hours ago on a pi4 without issue. Maybe try a different server?

11jwolfe2 commented 4 years ago

I am using Israel. What server are you using just so I can be as consistent as possible?

JB09 commented 4 years ago

I am using Israel. What server are you using just so I can be as consistent as possible?

Romania

haugene commented 4 years ago

FYI. Others are also having issues on armhf. I'll have a look this weekend, but until then it is possible to revert to 2.14 See if there are any common issues with #1428 and we can follow up there :+1:

bulfinch commented 4 years ago

I have been running this in Docker on Synology and with both :latest and :dev versions I'm now getting a crash when starting. This is from the logs

Using OpenVPN provider: PIA
Provider PIA has a custom startup script, executing it
Downloading OpenVPN config bundle openvpn-nextgen into temporary file /tmp/tmp.DcahjJ
Extract OpenVPN config bundle into PIA directory /etc/openvpn/pia
Modify configs for this container
Starting OpenVPN using config CA Toronto.ovpn
Setting OpenVPN credentials...
adding route to local network 192.168.1.1/24 via 172.17.0.1 dev eth0
RTNETLINK answers: Operation not permitted

I do have set --cap-add=NET_ADMIN

    --cap-add=NET_ADMIN \
    --device=/dev/net/tun \
    --log-opt max-size=10m \
    -d --restart always
    -v /volume1/docker/transmission-openvpn/resolv.conf:/etc/resolv.conf \
    -v /volume1/data/torrents:/data \
    -v /volume1/docker/transmission-openvpn:/config \
    -e "OPENVPN_PROVIDER"="PIA" \
    -e "OPENVPN_CONFIG"="CA Toronto" \
    -e "OPENVPN_USERNAME"="***" \
    -e "OPENVPN_PASSWORD"="***" \
    -e "GLOBAL_APPLY_PERMISSIONS"="true" \
    -e "TRANSMISSION_HOME"="/data/transmission-home" \
    -e "TRANSMISSION_WEB_UI"="/path/to/web/ui" \
    -e "TRANSMISSION_WEB_HOME"="true" \
    -e "CREATE_TUN_DEVICE"="true" \
    -e "LOCAL_NETWORK=192.168.1.1/24" \
    -e "ENABLE_UFW"="false" \
    -e "UFW_ALLOW_GW_NET"="false" \
    -e "UFW_EXTRA_PORTS"="" \
    -e "UFW_DISABLE_IPTABLES_REJECT"="false" \
    -e "DROP_DEFAULT_ROUTE"="" \
    -e "WEBPROXY_ENABLED"="true" \
    -e "WEBPROXY_PORT"="8888" \
    -e "OPENVPN_OPTS=--inactive 3600 --ping 10 --ping-exit 60" \
    -e "LOG_TO_STDOUT"="false" \
    -e "PGID=101" \
    -e "PUID=1026" \
    -p 8888:8888 \
    --name "Transmission-OpenVPN" \
    haugene/transmission-openvpn:latest

Thoughs?

jt196 commented 4 years ago

@bulfinch I changed the local network to 192.168.1.1/32 and that seems to have done the job.

bulfinch commented 4 years ago

I fixed the "crash" by changing -e "LOCAL_NETWORK=192.168.1.1/24" \ to -e "LOCAL_NETWORK=192.168.1.0/24" \ and removed the value of -e "OPENVPN_CONFIG"="" \ However now im seeing 

setting transmission port to 53722
localhost:9091/transmission/rpc/ responded: "success"
Checking port...
Error: portTested: http error 0: No Response
initial setup complete!

edit: I am able to get an open port using -e "OPENVPN_CONFIG"="Spain" \

nervling commented 4 years ago

I am running 5 instances of this on unraid and all of them started failing with:

Using OpenVPN provider: PIA Provider PIA has a custom startup script, executing it Downloading OpenVPN config bundle openvpn-nextgen into temporary file /tmp/tmp.GmkdLo Extract OpenVPN config bundle into PIA directory /etc/openvpn/pia Modify configs for this container Starting OpenVPN using config CA Toronto.ovpn Setting OpenVPN credentials... adding route to local network 192.168.86.0/24 via 192.168.86.1 dev eth0 RTNETLINK answers: File exists

Reverting to 2.14 fixed the issue for me. If I can provide further info to help debug, let me know!

jt196 commented 4 years ago

in docker replace (using your docker instance local IP):

-e "LOCAL_NETWORK=192.168.1.1/24" \

with

-e "LOCAL_NETWORK=192.168.1.1/32" \

docker compose:

replace 

        environment:
            ...
            - LOCAL_NETWORK=192.168.1.77/24
            ...

with 

        environment:
            ...
            - LOCAL_NETWORK=192.168.1.77/32
            ...

This worked for me and I had the same RTNETLINK message.

haugene commented 4 years ago

@bulfinch It sounds like you got it resolved. But just an FYI you should remove the --device=/dev/net/tun when you have "CREATE_TUN_DEVICE"="true". That being said, the "CREATE_TUN_DEVICE"="true" is now default and you can drop that too.

As for the issues with RTNETLINK. This is due to the iptables add command that tries to add a rule that already exists. I have yet to understand exactly why this differs on different hosts but apparently it does. This is a result of the "LOCAL_NETWORK" variable and changing it might fix it as it's proposed here.

But... Changing it to x.x.x.x/32 would only open for one single IP to access the web ui. You could read up on CIDR addresses to understand exactly how they work, but the /32 basically says how many bits are locked. For a local network setup 192.168.0.0/16 would allow anything inside the 192.168.x.x space and should be safe as this is not in the external IP ranges.

No matter if it works or not, I'm interested in seeing the ip tables setup you have when the container is running. Can you exec into the container and run ip route list and post the results here.

jt196 commented 4 years ago

This is mine:

0.0.0.0/1 via 10.15.11.5 dev tun0
default via 172.31.0.1 dev eth0
10.15.11.1 via 10.15.11.5 dev tun0
10.15.11.5 dev tun0 proto kernel scope link src 10.15.11.6
31.168.172.145 via 172.31.0.1 dev eth0
128.0.0.0/1 via 10.15.11.5 dev tun0
172.31.0.0/16 dev eth0 proto kernel scope link src 172.31.0.2
haugene commented 4 years ago

Allright. And just for a sanity check, are you experiencing issues now?

I am not an expert on this, but for those even less experienced with reading these: tun0 is the VPN interface, eth0 is the Docker interface - like it's normal "network card" in whatever sense we can speak about that in container terms.

Interpreting yours we see that we have two wide catching rules. The two first ones. It says that the default route is through Docker interface (without VPN) but that doesn't matter because we have the 0.0.0.0/1 that is a wide catch all with higher specificity (/1) and will therefore grab all packets and route them through VPN. The exceptions are other rules that are more specific than itself.

0.0.0.0/1 via 10.15.11.5 dev tun0 <- everything through vpn
default via 172.31.0.1 dev eth0 <- default on the regular interface)
10.15.11.1 via 10.15.11.5 dev tun0 <- guessing VPN provider is on 10.x and that this has to do with them
10.15.11.5 dev tun0 proto kernel scope link src 10.15.11.6 <- guessing VPN provider is on 10.x and that this has to do with them
31.168.172.145 via 172.31.0.1 dev eth0 <- not sure why this is there, but it would be tempting to say that you have given this in LOCAL_NETWORK?
128.0.0.0/1 via 10.15.11.5 dev tun0 <- Not sure what this does either. But it's going over the VPN interface, so it's OK. Might be provider specific.
172.31.0.0/16 dev eth0 proto kernel scope link src 172.31.0.2 <- The docker network. Other containers in the same network will be on 172.31.x.x

Hope that helps somehow.

jt196 commented 4 years ago

@haugene I've amended my Docker-compose.yml file with the old /24 setting, deleted and respun the container and it's back up and working now.

Thanks for the explanation of these ports, again, some work needed on my networking knowledge, this may very well help me understand a bit more!

bulfinch commented 4 years ago

@bulfinch It sounds like you got it resolved. But just an FYI you should remove the --device=/dev/net/tun when you have "CREATE_TUN_DEVICE"="true". That being said, the "CREATE_TUN_DEVICE"="true" is now default and you can drop that too.

As for the issues with RTNETLINK. This is due to the iptables add command that tries to add a rule that already exists. I have yet to understand exactly why this differs on different hosts but apparently it does. This is a result of the "LOCAL_NETWORK" variable and changing it might fix it as it's proposed here.

But... Changing it to x.x.x.x/32 would only open for one single IP to access the web ui. You could read up on CIDR addresses to understand exactly how they work, but the /32 basically says how many bits are locked. For a local network setup 192.168.0.0/16 would allow anything inside the 192.168.x.x space and should be safe as this is not in the external IP ranges.

No matter if it works or not, I'm interested in seeing the ip tables setup you have when the container is running. Can you exec into the container and run ip route list and post the results here.

Thank you for the reply. I'll read up on the CIDR addresses. I do have the container working, and I have removed the --device=/dev/net/tun (for testing, this was not the culprit but I understand what you are saying) Here is the containers ip route list 0.0.0.0/1 via 10.21.112.1 dev tun0 default via 172.17.0.1 dev eth0 10.21.112.0/24 dev tun0 proto kernel scope link src 10.21.112.3 128.0.0.0/1 via 10.21.112.1 dev tun0 172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.6 192.168.1.0/24 via 172.17.0.1 dev eth0 212.102.49.22 via 172.17.0.1 dev eth0

Thank you again for your work and help.

justin-peacock commented 4 years ago

Does anyone have this working on Synology yet? I'm still having crashing issues with it.

bulfinch commented 4 years ago

Does anyone have this working on Synology yet? I'm still having crashing issues with it.

If you look at my posts, I have it running on my Synology. Hopefully there's something useful there.

jt196 commented 4 years ago

@mrdink mine's ok too. Best you check out the logs and search here for answers. There are a couple of steps to get it running on the Syno - not sure whether you've got it up and running yet.

justin-peacock commented 4 years ago

Ok I think I got it working. I set the LOCAL_NETWORK to /32 and removed --device=/dev/net/tun

kperinga commented 4 years ago

Does anyone have this working on Synology yet? I'm still having crashing issues with it.

@mrdink I finally got mine working on Synology. It seems the transmission environment settings were the issue, so I deleted all of those from my compilation file, except for TRANSMISSION_HOME. Then the container started up and I re-did all my settings inside transmission itself - I've now had three instances of the container running without issue for the last three or four days.

Trial and error shows the containers work without issue with LOCAL_NETWORK set to either /24 or to /32, and with or without --device=/dev/net/tun. However, as soon as I set any of the Transmission environment variables in the compilation file the container won't start or at best starts up and crashes. I assume all this is due to the compatibility issues mentioned by @haugene

jt196 commented 4 years ago

@haugene please cancel that last comment about it working, I got my containers mixed up. v3 IS NOT working unless I run the /32 local network setting. Doesn't get past booting up. Weirdly in Portainer, the logs switch on, then off, off/on, until they just disappear.

jt196 commented 4 years ago

And FYI, - LOCAL_NETWORK=192.168.1.1/16 doesn't work either - same results.

chrishilbert commented 4 years ago

I am still getting auth_failed. Not sure what I've done wrong. Seems like it's not using my username/password?

chrishilbert commented 4 years ago

If I try to use OPENVPN_CONFIG=nextgen/ or OPENVPN_CONFIG=openvpn-nextgen/ it's not found. No prefix on the config path the config is found. If I try my control panel u/p with the working config path, I get auth_failed. Stumped.

haugene commented 4 years ago

@chrishilbert As long as you've pulled the latest image you have the nextgen servers. There is no need to prefix anything. The username/password that you pass in is written to the file /config/openvpn-credentials.txt. Can you exec into the container and check it's content? Is it the same as you pass in or has something happened while evaluating the environment variable?

If you can't access it before it crashes you can try to mount a folder to /config in the container and then check after a run. The file should then be there.

kennethkkk commented 4 years ago

Suddenly can not connect anymore:

Using OpenVPN provider: PIA Provider PIA has a custom startup script, executing it Downloading OpenVPN config bundle openvpn-nextgen into temporary file /tmp/tmp.moIMbe Extract OpenVPN config bundle into PIA directory /etc/openvpn/pia Modify configs for this container Starting OpenVPN using config AU Sydney.ovpn Setting OpenVPN credentials... . . Wed Nov 4 11:06:27 2020 AUTH: Received control message: AUTH_FAILED Wed Nov 4 11:06:27 2020 SIGTERM[soft,auth-failure] received, process exiting

wickning1 commented 4 years ago

I switched to the dev image to see if it would fix port forwarding and also have the AUTH_FAILED issue. I checked openvpn-credentials.txt and it looks fine. It has my username on the first line and my password on the second line.

wickning1 commented 4 years ago

latest is giving me AUTH_FAILED as well. latest-alpine is fine.

jt196 commented 4 years ago

Mine has broken in the last day too - seems like it's stuck in a start up loop. Similar to @kennethkkk's log I'd say, I try to exec into the container and it says it's restarting, so it's hard to say what the problem is. Think I'll see if I can get AirVPN set up at this rate...

chrishilbert commented 4 years ago

Mine has broken in the last day too - seems like it's stuck in a start up loop. Similar to @kennethkkk's log I'd say, I try to exec into the container and it says it's restarting, so it's hard to say what the problem is. Think I'll see if I can get AirVPN set up at this rate...

Change your VPN location. Toronto seems hosed

jt196 commented 4 years ago

Israel, Romania... not sure I really want to play whack-a-mole with the Next gen servers until none of them work...