Closed qdm12 closed 3 years ago
Thanks for looking into this!
Same here. Last bit of output from $ docker-compose up vpn
:
vpn_1 | 2020-07-11T11:23:37.050-0700 INFO ip getter: Public IP address is ###.###.###.###
...
vpn_1 | 2020-07-11T11:23:42.226-0700 INFO port forwarding: port forwarded is #####
vpn_1 | 2020-07-11T11:23:42.226-0700 INFO port forwarding: writing forwarded port to /forwarded_port
vpn_1 | 2020-07-11T11:23:42.226-0700 INFO firewall configurator: accepting input traffic through tun0 on port #####
Then in the running container;
bash-5.0# nc -vl -s 0.0.0.0 -p #####
listening on 0.0.0.0:##### ...
Then on the docker host:
# nc -vz ###.###.###.### #####
No further output from either nc
command.
I'm looking into it. Probably a firewall issue, although I'm having trouble pinning it down.
Was it working for you before perhaps?
Actually trying with -e FIREWALL=off
and it still doesn't work. I manually checked the firewall was indeed disabled and allowing everything. Maybe we're not trying it right? Another user reported it was working in his torrent client 😕
Are you able to test it with i.e. pia own client? What ip address do you use to test it? The public one?
Thanks guys
I use the IP reported as the public IP in the output from the entrypoint. I also confirmed that IP was correct with # wget -qO- https://ipinfo.io
in the running container. I also tested with FIREWALL=off
and it still doesn't work. I'm going to try some other tests today.
I can't get port forwarding working with the desktop GUI client either. I opened a support ticket with the following:
I'm unable to get port forwarding working under any of 3 methods. Most recently I tried it with the GUI desktop client. I ran the pia-linux-2.2.1-05193.run
installer:
$ ~/Downloads/pia-linux-2.2.1-05193.run
Verifying archive integrity... 100% MD5 checksums are OK. All good.
Uncompressing Private Internet Access 100%
=================================
Private Internet Access Installer
=================================
[sudo] password for rpatterson:
✔ Added group piavpn
✔ Added group piahnsd
✔ Copied Private Internet Access files
✔ Allow non-root /opt/piavpn/bin/pia-unbound to bind to privileged ports
✔ Created var folder
✔ Installed icon
✔ Created desktop entry
✔ Added piavpnrt routing table
✔ Added piavpnOnlyrt routing table
✔ Added piavpnWgrt routing table
✔ Set wgpia interface to be unmanaged
✔ Created piavpn service
Created symlink /etc/systemd/system/multi-user.target.wants/piavpn.service → /etc/systemd/system/piavpn.service.
✔ Started piavpn service
I logged in with my credentials, enabled port forwarding, connected to the VPN, and copied the public IP and forwarded port. Then I started listening on the forwarded port:
$ nc -vn -l -s 0.0.0.0 -p 38971
Listening on 0.0.0.0 38971
Finally, I tried connecting to that port on the public IP, but the connection is blocked:
$ nc -vz 172.83.40.103 38971
nc: connect to 172.83.40.103 port 38971 (tcp) failed: Connection refused
I have also tried this with a standard OpenVPN configuration on the command line using both of the following approaches:
I've also tried it with this Docker image:
https://github.com/qdm12/private-internet-access-docker/issues/177
None of them have worked.
Using this actually works. But nc, nmap, telnet don't.
When running nmap -p <PORT> <IP>
, I get that the port is in a filtered
state.
My guess is private internet access filters packets and most won't make it, not sure why.
Although somehow https://www.yougetsignal.com/tools/open-ports/ works, but I can't tell how they do it.
Let's keep the issue opened. Once you have a resolution, please post it back here and I'll update #53 and the wiki with how to check.
Thanks for the debugging!!
I tried that approach using ncat
instead of nc
:
$ ncat -v -l 38971
Ncat: Version 7.80 ( https://nmap.org/ncat )
Ncat: Listening on :::38971
Ncat: Listening on 0.0.0.0:38971
But neither local ncat
on the connecting side nor that web site worked:
$ ncat -vz 172.83.40.113 38971
Ncat: Version 7.80 ( https://nmap.org/ncat )
Ncat: Connection refused.
Port 38971 is closed on 172.83.40.113.
This is still using the GUI desktop client.
I also installed transmission-gtk
, fired it up, changed the port to the one copied from the PIA GUI client, and clicked the test open/closed button in the Transmission GUI and it still doesn't work. Safe to say, the issue is not your Docker image. :-/
This is also not working for me anymore, and it's only a recent version of the container that broke it. Here is my docker-compose I'm using.
pia:
restart: always
container_name: pia
image: qmcgaw/private-internet-access:latest
ports:
- 8888:8888/tcp
- 8081:8081/tcp
- 5800:5800/tcp
- 7878:7878/tcp
- 9117:9117/tcp
- 9089:8989/tcp
- 6789:6789/tcp
- 6767:6767/tcp
environment:
- REGION=CA Toronto
- USER=xxxxxxxx
- PASSWORD=xxxxxxxx
- FIREWALL=off <--- turned off as an experiment, no effect.
- EXTRA_SUBNETS=192.168.25.0/24,192.168.99.0/29
devices:
- /dev/net/tun
cap_add:
- NET_ADMIN
@oester your issue is irrelevant to this issue about vpn server side port forwarding. Please refer to #190. It should be fixed now (pushed a commit a few minutes ago).
Hrm - tried pulling latest after your comment, still fails. I'll have to dig into it. Dropped back to :v2 - that works. Will try and debug later.
@oester please let's continue this on #194. There is indeed an issue with port publishing, I'm working on it.
@oester the port publishing issue is now fixed see #194 if you're curious, if not pull the latest image 😉 I'll make a release soon enough as :v3
is long overdued, I can't even break :latest
no more 😄
Confirming that the issue is now fixed in the latest build.
The commit that claimed to close this issue isn't actually relevant to this issue. However, I've since discovered that the PIA forwarded port does seem to start working using this container eventually. When I first fire up the container, get the reported forwarded port, set that as the peer-port
in Transmission, and test whether the port is open in a Transmission UI (either web, or the native GTK remote UI), it reports as closed. After some amount of time, however, it starts reporting as open. So I'm not sure what's going on there, maybe some sort of rate limiting while I was iterating through getting my docker-compose.yml
working, but at any rate, and again, it's certainly not related to this image.
Ugh, the support responses to my PIA ticket are terrible, repeatedly not at all reading the details I report. At any rate, this reply seems possibly relevant so I thought I'd share it in case it helps anyone else in the future, though I'm not sure what it means:
"Listening ports are only open while data is actively moving through them." You would need to set this up, then when data is moving, you can run the test.
@rpatterson Yes the commit discussed was for another issue which sounded like the issue title for another user 😉
I believe it is heavily filtered, maybe only peer-to-peer traffic goes through (although p2p uses tcp and udp...).
From the answer they gave, maybe... maybe, they have some deep packet inspection in their gateway / firewall which would activate only the port forwarding if enough data is sent to you (i.e. torrenting). That might make sense to limit the locking of forwarded port if the user requested it but does not use it, as there are only 65535 ports per ip.
Although I just ran something:
nc -l -p 49654
cat test.mkv | nc -vv -n -w 600 <vpn ip> 49654
But still get a timeout error, so I guess it is just some deep packet inspection allowing only peer to peer shaped traffic. Sending a movie directly through it won't work I think.
Again, that youseeme port checking works and I still have no clue how.
@rpatterson
the support responses to my PIA ticket are terrible, repeatedly not at all reading the details I report.
agreed, their support team needs more coffee.
"Listening ports are only open while data is actively moving through them."
I'm surprised they wrote this, because that's not true: once you obtain the forwarded port, that's open, full stop. I've had plenty of cases in which traffic on the port was non existent for hours, yet I was still connectable from outside.
@qdm12
I believe it is heavily filtered, maybe only peer-to-peer traffic goes through (although p2p uses tcp and udp...).
that is not the case, I have tested PF successfully with webservers, ssh servers, etc.
I guess it is just some deep packet inspection allowing only peer to peer shaped traffic
lol, that would be a VPN provider using DPI for exactly the opposite of its classic use! jokes aside, they don't do that.
Ha who knows what the vpn service provider do on their end. Let's not trust anyone 😄
@piramiday If you managed to have port forwarding work using the pia official client, maybe you can help out @rpatterson debug why it doesn't work with its official pia client? @rpatterson have you got any news from pia about it? That way we can clarify what's wrong with my container (or if it's not the container).
Even though I have 0 use for it, I would love to fix port forwarding for pia. Also apparently port forwarding for vyprvpn seems to work which makes it even stranger (I need to test it personally though to make sure).
So interesting note here. I've been setting up port forwarding for Vyprvpn successfully but it won't work with pia. I use the following, assuming the port forwarded is 9999
and the vpn ip is 55.55.55.55
:
iptables -A INPUT -i tun0 -p tcp --dport 9999
(verified with iptables -nvL
)docker exec -it gluetun nc -l -v -p 9999 0.0.0.0
On another host:
docker run -it --rm alpine:3.12
echo -n "blabla" | nc 55.55.55.55 9999
exit
On Vyprvpn it works without issue, and still nothing for pia. I'll add it for another provider (#102 for example) to ensure this is a pia related issue, but it looks like it so far.
I've had some issues with my scripts as well (unrelated to docker) in the past few days with the canadian port-forwarding servers -- @rpatterson try using Switzerland or France, instead.
I tried (with gluetun) in France and Switzerland and it still doesn't work. Piramiday, what's your host os where you have working port forwarding? Fingers crossed it's some Linux or osx 😄
have you tried testing port forwarding with PIA outside of docker? I'm running the official PIA app for ubuntu-based distros, obtaining a forwarded port via the command-line script.
Could you print out the routing table you get when port forwarding is on with the pia app? (use ip route
usually). For now I just use the container. I believe there should be some bad route or missing route for the container, let's see.
Hi,
I just yesterday started to have issues with Sweden region. It wasn't just port forwarding. It didn't connect to the region at all. Seems to be that the migration to the new next generation network at Private Internet Access is the bandit in my case. It just got out of beta It started working when I switched to DE Frankfurt. Not sure if it's the same for you?
I think for it to work properly it would have to connect to the next generation servers instead of the old ones. The new servers are listed here: https://serverlist.piaservers.net/vpninfo/servers/new
@Erijoda I updated the IP addresses for PIA this morning, so it might work (maybe still using old servers).
Where did you get the https://serverlist.piaservers.net/vpninfo/servers/new url? Is this some official PIA API? Can you please reply on #216 as this is irrelevant to this issue.
I think for it to work properly it would have to connect to the next generation servers instead of the old ones. The new servers are listed here: https://serverlist.piaservers.net/vpninfo/servers/new
I've had problems with PIA port forwarding in Europe also last few weeks, the container does not receive a open port. However according to the first sentence in bold from yesterday on this link: https://www.privateinternetaccess.com/helpdesk/kb/articles/can-i-use-port-forwarding-without-using-the-pia-client-current-gen-only
Only the old servers support acquiring a forwarded port without the PIA client software.
Ah that sucks! I was just modifying the code to replace old servers by new ones and was thinking to myself, sweet an almost drop in replacement. I guess I'll update & keep both until port forwarding is available on the new servers...
Sounds good, i just randomly stumbled upon it after trying to forward a port without this container and could not make it work with the new config files. Within the container i have switched between all supported countries in the last week or so, now only Romania is working with the gluetun container. Hopefully you can now resolve the issue until the new servers have this functionality too!
-e VPNSP="private internet access old"
Actually please try with Docker image tag :new-pia-servers
, that would help me 👍 All the regions should work for new and old servers, hopefully port forwarding reworks again on older servers too.
Sounds great, I'll update my compose file later tonight!
FYI I just re-updated :new-pia-servers
with a bunch of fixes, port forwarding seems to work too (built for amd64 only).
On the other hand, would you please let me know how you use port forwarding? That's the whole point of this issue, I and some others cannot figure out how to test it works. Listening on the forwarded port with nc -l :<port>
does not seem to work for example. Thanks!
I use the port forwarding with ruttorent. I run rutorrent with pia together using docker-compose. I have a shell script that picks up the port forwarding from pia and then applies it to the rutorrent settings. In the rutorrent console it shows a little green button when the port is opened.
You can try login into the container and use nc -zv localhost (or the public ip from pia) {port_number} command (if nc is installed) to check if the port is open or not.
Another option would to have nmap installed and try out a local scan.
I had issues in the past but for a while my port forwarding works well. I read previous conversations about port forwarding not working with some European servers. I am always connected to Berlin or Frankfurt and both location seem fine to me.
I am happy to share my docker compose file and the shell script which does the configuration with you.
Regards,
On Fri, 28 Aug 2020, 00:15 Quentin McGaw, notifications@github.com wrote:
FYI I just re-updated :new-pia-servers with a bunch of fixes, port forwarding seems to work too (built for amd64 only).
On the other hand, would you please let me know how you use port forwarding? That's the whole point of this issue, I and some others cannot figure out how to test it works. Listening on the forwarded port with nc -l :
does not seem to work for example. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qdm12/gluetun/issues/177#issuecomment-682237846, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRLJXJQ3L2YYPCNSLSMF3SC3SJTANCNFSM4OKTILNA .
Hello groenator, I just tried again with region CA Vancouver
, where forwarded port is 49806
and VPN public IP is 172.83.40.25
, running
docker run exec -it gluetun nc -lv -p 49806 0.0.0.0
Then on my host (not tunnelled):
docker run -it --rm alpine:3.12 nc -zv 172.83.40.25 49806
But it still doesn't work. Installing apk add netcat-openbsd
doesn't help either, with error
nc: connect to 172.83.40.25 port 49806 (tcp) failed: Connection refused
Can you please try on your end? My PIA subscription is expiring like today I think haha, so it would be nice if we can sort this issue out today too 😄
Sure! I will try out and let you know. Do you want me to use the same zone or use a server from Europe? I usually connect to Germany.
Which tag image should I use?
Regards,
On Sat, 29 Aug 2020, 14:52 Quentin McGaw, notifications@github.com wrote:
Hello groenator, I just tried again with region CA Vancouver, where forwarded port is 49806 and VPN public IP is 172.83.40.25, running
docker run exec -it gluetun nc -lv -p 49806 0.0.0.0
Then on my host (not tunnelled):
docker run -it --rm alpine:3.12 nc -zv 172.83.40.25 49806
But it still doesn't work. Installing apk add netcat-openbsd doesn't help either, with error
nc: connect to 172.83.40.25 port 49806 (tcp) failed: Connection refused
Can you please try on your end? My PIA subscription is expiring like today I think haha, so it would be nice if we can sort this issue out today too 😄
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qdm12/gluetun/issues/177#issuecomment-683293063, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRLJUMYVKTDQYFOUM75YTSDEB2XANCNFSM4OKTILNA .
You can use any region with the latest
Docker image tag. Maybe I'm wrong with my commands but it should be working I think.
Thanks to @groenator we have tested things a bit further.
Assuming your VPN IP address is 100.100.100.100
and forwarded port is 5000
;
We run another container with
docker run -it --rm alpine:3.12
apk add --no-cache -X http://dl-cdn.alpinelinux.org/alpine/edge/main nmap
nmap -sS -p 5000 100.100.100.100
opened
port 🎉 nc -l 0.0.0.0 5000
in gluetun, the result is a filtered
port 😢 Conclusion: Port forwarding seems to work for certain applications. Only explanation coming to my mind is still deep packet inspection on the incoming port on the vpn side (not on your vpn connection though AFAIK). It might make sense they have that to prevent some attacks, but it really sucked hours of my (and others) time trying to figure it out, which sucks.
hi all, I hadn't time to experiment again with this. I wanted to add:
deep packet inspection on the incoming port on the vpn side
again, nope -- they are the guys fighting DPI.
I had another chat with PIA support -- allow me to summarize what I got in this table, that might be helpful:
protocol | network | PF via App | PF via API |
---|---|---|---|
OpenVPN | current | yes | yes |
OpenVPN | nextgen | yes | no |
WireGuard | current | no | no |
WireGuard | nextgen | yes | no |
where, of course:
curl
request to 209.222.18.222:2000.their reply in full:
Hi piramiday, thanks for sharing this.
I'm aware of the port forwarding limits of the next gen, it's already in place in the docker image actually 😉
It's good they have plans to support it with an API in the future as a few of us seem to stay on VPNSP=private internet access old
to keep that forwarded port for now.
they are the guys fighting DPI
Anyway, back to the issue, if one day you can manage to run an http server (or iperf for example) listening on your client and access it with the VPNip:forwardedPort then please let us know 👍
@piramiday thanks, although that's for the newer v4 servers. Do you mind however re-posting that message on #242 as I'm actively (well sort of) working on it, so it's definitely useful for the conversation on that issue. Cheers!
Am I wrong, or doesn't port forwarding for PIA work at all anymore?
2020-10-26T08:21:04.118Z INFO dns configurator: using DNS address 127.0.0.1 internally
2020-10-26T08:21:04.118Z INFO dns configurator: using DNS address 127.0.0.1 system wide
2020-10-26T08:21:04.333Z INFO unbound: init module 0: validator
2020-10-26T08:21:04.333Z INFO unbound: init module 1: iterator
2020-10-26T08:21:04.366Z INFO unbound: start of service (unbound 1.10.1).
2020-10-26T08:21:04.848Z INFO unbound: generate keytag query _ta-4a5c-4f66. NULL IN
2020-10-26T08:21:06.653Z INFO dns over tls: DNS over TLS is ready
2020-10-26T08:21:07.566Z INFO VPN gateway IP address: 10.60.110.1
2020-10-26T08:21:08.580Z INFO You are running on the bleeding edge of latest!
2020-10-26T08:21:10.412Z INFO ip getter: Public IP address is 172.98.71.62
2020-10-26T08:21:12.568Z ERROR port forwarding: cannot obtain token: Get "https://p5140218:***@10.0.0.1/authv3/generateToken": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2020-10-26T08:21:12.568Z INFO port forwarding: Trying again in 10s
2020-10-26T08:21:27.569Z ERROR port forwarding: cannot obtain token: Get "https://p5140218:***@10.0.0.1/authv3/generateToken": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2020-10-26T08:21:27.569Z INFO port forwarding: Trying again in 10s
2020-10-26T08:21:37.286Z WARN Caught OS signal terminated, shutting down
2020-10-26T08:21:37.286Z WARN ip getter: loop exited
2020-10-26T08:21:37.286Z INFO Clearing ip status file /tmp/gluetun/ip
2020-10-26T08:21:37.286Z WARN http server: context canceled: exiting loop
2020-10-26T08:21:37.286Z WARN openvpn: context canceled: exiting loop
2020-10-26T08:21:37.286Z WARN port forwarding: loop exited
2020-10-26T08:21:37.286Z INFO Clearing forwarded port status file /tmp/gluetun/forwarded_port
2020-10-26T08:21:37.286Z WARN dns over tls: context canceled: exiting loop
2020-10-26T08:21:37.289Z WARN dns over tls: loop exited
2020-10-26T08:21:37.315Z WARN openvpn: loop exited
2020-10-26T08:21:37.786Z WARN http server: loop exited
2020-10-26T08:21:37.786Z INFO Shutdown successful
Please create another issue, this issue is about having a reliable way to test port forwarding.
I'll increase the http round-trip timeout, which seems to be the problem here. (see #271 )
Guys now that PIA moved to their pia v4 servers and that port forwarding changed and is supported in this image, can you please try the following see if it works?
Launch gluetun with PORT_FORWARDING=on
and container name gluetun
We assume the port forwarded is 9999
and the vpn ip is 55.55.55.55
Just make double-make-sure your port forwarded is allowed in the firewall:
docker exec gluetun iptables -A INPUT -i tun0 -p tcp --dport 9999
Alternatively run the container with FIREWALL=off
.
Run a netcat server on your forwarded port 9999
docker exec -it gluetun nc -l -v -p 9999 0.0.0.0
You can press CTRL+C to stop it later when you're done with the below steps.
In another shell out of Gluetun/VPN, run
docker run -it --rm alpine:3.12
echo -n "blabla" | nc 55.55.55.55 9999
exit
Does that work or still hang???
@qdm12
I ran above commands with the forwarded ports. When you ask if it hangs or work are you referring to the commands run on alpine? If so, it take around 10s but does not show anything and moves to the next line where I enter exit.
Attaching compose command. Let me know if you want me to run it with modifications. Fair warning that I am new to all this but I can test stuff for you if it's just running commands.
gluetun:
image: qmcgaw/private-internet-access
container_name: gluetun
cap_add:
- NET_ADMIN
ports:
- 8888:8888/tcp # Tinyproxy
- 8388:8388/tcp # Shadowsocks
- 8388:8388/udp # Shadowsocks
- 8000:8000/tcp # Built-in HTTP control server
volumes:
- ./app/gluetun:/gluetun
environment:
# PIA Specific settings
- VPNSP=private internet access
- PROTOCOL=udp
- USER=${PIA_USERNAME}
- PASSWORD=${PIA_PASSWORD}
- REGION=${PIA_SERVERNAME}
- PIA_ENCRYPTION=strong
- PORT_FORWARDING=on
# DNS
- DOT=on
- DOT_PROVIDERS=cloudflare
# User Specific
- TZ=${PREFERRED_TZ}
- UID=${PUID}
- GID=${PGID}
# Allow for example your LAN, set to: 192.168.1.0/24
restart: always
Thanks @nviraj for trying it out! Yeah it shouldn't hang for more than a second and should print out something.
I guess the state of PIA port forwarding is still the same then 😕
I'm still so confused why it works with certain applications like torrenting but not, say, an http server.
I added instructions on how to test port forwarding in the Wiki here. Closing this issue for now, as it works for other VPN providers. If it doesn't for PIA, it's a PIA related issue I guess, as the firewall setup is common to all providers.
TLDR: Port forwarding with pia seem to not work
Is this urgent?
What VPN service provider are you using?
What's the version of the program?
See the line at the top of your logs
Running version latest built on 2020-06-22T00:21:46Z (commit 3b43b7c)
What are you using to run the container?
Extra information
Port is closed at: