qbittorrent / qBittorrent

qBittorrent BitTorrent client
https://www.qbittorrent.org
Other
27.04k stars 3.89k forks source link

4.2.2 download/upload stalled for all torrents using only TCP with VPN (µTP works) #12253

Closed gothicserpent closed 4 years ago

gothicserpent commented 4 years ago

qBittorrent version and Operating System

4.2.2 / Windows 10 Enterprise 10.0.18362 Build 18362

What is the problem

4.2.1 download/upload working for me, i tried to update to 4.2.2. after i updated, the download/upload stopped working (stalled). For my network settings (using a VPN), I clicked on Connection -> Enabled Protocol: "TCP" which does not work. Tried to change the port in settings of 4.2.2, still didn't work. Connection -> Enabled Protocol: "µTP" or "TCP and µTP" DOES work (implying µTP works but not TCP because when "TCP and µTP" is selected it chooses µTP). After I downversioned back to 4.2.1 the dl/ul immediately started working again with TCP only.

What is the expected behavior

Expected download/upload to function

Steps to reproduce

download / install qbittorrent 4.2.2 version download mullvad vpn from mullvad.net, sign up for trial (~$5.80) (you might be able to use another VPN such as NordVPN, ExpressVPN or ProtonVPN as well; the steps below are mullvad specific)

Open the Windows Control Panel. Click on Network and Sharing Center. Click on Change adapter settings. On the adapter that contains "TAP-Windows Adapter V9" in its name, right click on it, select Rename, and enter Mullvad. Proceed with the steps below

Open qBittorrent. Click on Tools. Click on Options. Click on Advanced. Change Network Interface to Mullvad if you use OpenVPN or wg-mullvad if you use WireGuard. (If you use another VPN, select that interface here.)

image

Click on OK and restart qBittorrent. Continue with the steps in the next section.

Click on Tools. Click on Options. Click on BitTorrent. Check Enable anonymous mode. Uncheck (disable) Enable DHT. Uncheck (disable) Enable PeX. Uncheck (disable) Enable Local peer discovery.

image

Click on Connection. For Enabled Protocol, use the drop-down menu to select TCP.

image

Make sure internet connection is enabled and sending and receiving packets. Enable Mullvad VPN in the taskbar. Attempt to test download any .torrent or magnet torrent link with data, which would otherwise work on another client or in QBittorrent version 4.2.1. There is no upload or download activity on the network at this point.

image

At this point, you're at the configuration I had working with 4.2.1 and that does not work with 4.2.2.

Extra info(if any)

If anyone else has this issue or you can reproduce it I hope you can fix it. Love your program. Thanks!

IMAGE OF DL/UL WORKING IN 4.2.1 using a VPN with TCP network setting (tracker details and file details hidden to prevent leaking)

4 2 1

IMAGE OF DL/UL NOT WORKING IN 4.2.2 using a VPN with TCP network setting (tracker details and file details hidden to prevent leaking)

4 2 2

allen-liaoo commented 4 years ago

I'm having the same issue with v4.2.2 right now. None of my trackers are working (sometimes their status are "Updating", sometimes "Not contacted", and other times "Not working") I've tried several torrent files and tracker links. All the same

gothicserpent commented 4 years ago

I'm having the same issue with v4.2.2 right now. None of my trackers are working (sometimes their status are "Updating", sometimes "Not contacted", and other times "Not working") I've tried several torrent files and tracker links. All the same

@AlienIdeology glad to see i'm not the only one. you're using a VPN right? did you enable anonymous mode in bittorrent options?

gothicserpent commented 4 years ago

Wanted to provide an update.

When I set Options -> Connection -> Enabled Protocol: TCP and µTP instead of TCP like I had before:

image

the upload and download connection is working again:

image

Perhaps there's an issue with the way TCP is handled in 4.2.2 that is different from how TCP is handled in 4.2.1 for users with a VPN?

@AlienIdeology you might try setting Enabled Protocol: TCP and µTP to see if it works?

allen-liaoo commented 4 years ago

@gothicserpent yeah I'm using VPN. I already have the protocol set to TCP and uTP. None of the trackers are working, sadly

allen-liaoo commented 4 years ago

oh i take it back, it is now working, but very slowly. (I didn't change anything, it fixed itself)

gothicserpent commented 4 years ago

oh i take it back, it is now working, but very slowly. (I didn't change anything, it fixed itself)

Cool - may have been an issue with the trackers you have. i was going to say that these are the only other settings I made:

image

and setting the network interface to my VPN in the advanced settings.

If anyone else has this issue I hope they report!

ghost commented 4 years ago

This is a libtorrent issue. Maybe @arvidn can better understand it.

arvidn commented 4 years ago

what version of libtorrent? I'm a bit confused by the title. If TCP-only doesn't work, how can both TCP and uTP be working? Does it mean to say "TCP over VPN does not work, uTP over VPN works"?

ghost commented 4 years ago

1.2.5 I think he meant TCP only mode doesn’t work. Which implies that only uTP is working for VPN. This might be related with https://github.com/arvidn/libtorrent/issues/4329

gothicserpent commented 4 years ago

Hi, yes I was able to get TCP working in 4.2.1 with my mullvad vpn, but when I tried 4.2.2 with VPN over TCP only, it did not have a connection (selected item highlighted in red does not work with a vpn):

image

It connected when i used the option "TCP and µTP". I believe the µTP connection was being utilized and TCP wasn't, so it was able to work over µTP only, in 4.2.2.

I am (assuming) maybe a change in the codebase was made that changed how TCP works with the VPN, such that it is no longer functional moving from 4.2.1 to 4.2.2. This is my speculation.

Rootax commented 4 years ago

1.2.5 I think he meant TCP only mode doesn’t work. Which implies that only uTP is working for VPN. This might be related with arvidn/libtorrent#4329

If this is related, it's not a vpn issue, but a problem with selecting a specific interface.

ghost commented 4 years ago

@arvidn I just reproduced this issue with a VPN.

Selecting VPN interface breaks TCP connection. You get uTP connections instantly as soon as you switch to uTP only.

*Binding to any interface(default option) while using a VPN doesn't cause this issue.

Trackers & DHT are working fine.

Selecting specific interface without a VPN does NOT cause this issue.

Could this be a problem in the network selection code of qBT?

rwasef1830 commented 4 years ago

I just saw this issue with qBittorrent 4.2.2 on Ubuntu 16.04 amd64 compiled with libtorrent 1.2.5. client_test is working. enum_if is still not seeing the gateway IPs. I will email you the results of those tools. @arvidn

I have TCP and uTP enabled, DHT, PEX, Local Peer discovery unchecked. Anonymous mode unchecked.

All torrents sit stalled and no trackers are contacted (all of them "Not Working").

EDIT: Recompiling qBittorrent with libtorrent-1_2_3 tag = zero issues.

RedLion76 commented 4 years ago

Same here. I am running Ubuntu 19.10. with qBit 4.2.2.

There was an update of libtorrent rasterbar yesterday, from v1.2.3 to v.1.2.5. SInce then, TCP works no more. TCP and µTP however works, but rather slow.

I can confirm rwasef1830s' post: Selecting "Any interface" from the dropdown menu seems to solve the issue.

I think libtorrent - rasterbar needs to be fixed.

Regards

RedLion

gothicserpent commented 4 years ago

SInce then, TCP works now more. TCP and µTP however works, but rather slow. @RedLion76 you mean TCP works no more?

RedLion76 commented 4 years ago

Yes. Sorry, typo.

arvidn commented 4 years ago

I cannot reproduce this. I need someone's help to understand whether there is some other configuration option that affects this. I'm testing current RC_1_2, which is pretty similar to libtorrent-1.2.5. At least I can't think of any changes that could have affected this.

This is what I've tried:

./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s .
./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s . --listen_interfaces=nordlynx:4325
./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s . --listen_interfaces=utun0:4325

That's testing without a VPN, with Private Internet Access (OpenVPN) and NordVPN's wireguard. In all cases I connect to peers over TCP and start downloading. Is there some other setting that could affect this?

Aerozolic commented 4 years ago

I have issues after updating to 4.2.2 as well. One issue is that if I have a specific interface (tun) selected then I get red connection status icon. And even if I select "Any interface" then I still can't get qbittorrent back to being connectable (showing yellow icon). Canyouseeme.org shows "connection refused" but if I try Transmission with the same port then it shows that the port is open. Firewall exception for qbittorrent is added and I did codesign as well like I do after every qbittorrent update. VPN connection was fine and is fine right now but there's definitely something wrong with qbittorrent. Using macOS Catalina 10.15.4

FranciscoPombal commented 4 years ago

@arvidn

I cannot reproduce this. I need someone's help to understand whether there is some other configuration option that affects this. I'm testing current RC_1_2, which is pretty similar to libtorrent-1.2.5. At least I can't think of any changes that could have affected this.

This is what I've tried:

./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s .
./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s . --listen_interfaces=nordlynx:4325
./client_test --enable_outgoing_utp=0 --enable_incoming_utp=0 -s . --listen_interfaces=utun0:4325

That's testing without a VPN, with Private Internet Access (OpenVPN) and NordVPN's wireguard. In all cases I connect to peers over TCP and start downloading. Is there some other setting that could affect this?

It is possible the issue is on qBIttorrent's side - specifically, most likely something is broken with the way it binds to interfaces. Can you think of the way this could be done wrong such that it would result in this problem?

gothicserpent commented 4 years ago

Just wanted to add this in: image To get the issue on my end I'm setting the interface to my VPN (which is configured with OpenVPN. not wireguard) / ip address to bind to: ipv4.

@arvidn You might try using your NordVPN config with OpenVPN and not wireguard and see if that makes a difference in testing TCP fails. I found some guides here on the OpenVPN setup for NordVPN at the bottom of this web site : https://support.nordvpn.com/General-info/1047408082/What-is-OpenVPN.htm

Edit: Saw online that Private Internet Access is it's own VPN. if you set that up with OpenVPN it should match my setup (assuming you configured utun0 to use your PIA VPN which you did I would assume -- i saw that this was a generic adapter reference rather than a vpn reference but if you bound it properly it should work). I would maybe try with NordVPN / OpenVPN.

the only settings I can think of from https://www.libtorrent.org/reference-Settings.html is --mixed_mode_algorithm=prefer_tcp since it's peer_proportional by default, although i suppose your boolean settings --enable_outgoing_utp=0 --enable_incoming_utp=0 you made will always prevent utp regardless.

the only other changes in my config as I have it is in the Bittorrent -> Privacy settings for DHT / etc. (which i disabled) and anonymous mode which i turned on: image

namely --max_pex_peers=0 --anonymous_mode=1 (I read that setting anonymous_mode to true will disable DHT and local peer discovery)

another thing i looked at was outgoing_interfaces, maybe look at if the default is correct.

FranciscoPombal commented 4 years ago

Just wanted to add this in: image To get the issue on my end I'm setting the interface to my VPN (which is configured with OpenVPN. not wiregaurd) / ip address to bind to: ipv4.

@arvidn You might try using your NordVPN config with OpenVPN and not wireguard and see if that makes a difference in testing TPC fails. I found some guides here on OpenVPN at the bottom of the page : https://support.nordvpn.com/General-info/1047408082/What-is-OpenVPN.htm . I hope this helps, I can't be of much more assistance because I have a win64 system and am using the QBittorrent windows 64 binary version.

What if you use All IPv6 addresses or All addresses as the address to bind to? Do you still get the issue?

gothicserpent commented 4 years ago

image

image

image

@FranciscoPombal I tried All addresses and TCP protocol and yes I still get this (no connection upload or download) - and the trackers have some errors here. The reason i switched to IPv4 earlier was to lower the rate of tracker error.

image

image

Edit: Also tried All IPv6 addresses, same result so far.

ghost commented 4 years ago

The issue occurs when you select the VPN interface to bind to. The IP address to bind to feature doesn't seem have any effect on this. It works fine when you select the VPN IP address to bind to but keep the network interface to all.

So basically when you're selecting the VPN interface qBt is failing to open up listen sockets for TCP. I don't believe this a problem on qBt end. Otherwise it would fail to open TCP sockets without a VPN as well. It probably has something to do with the changes that made through in libtorrent 1.2.4. Specially regarding VPNs/SOCK5 proxy/Multihoming improvements.

arvidn commented 4 years ago

@FranciscoPombal how do those settings translate to the libtorrent listen_interfaces setting?

The fact that they are presented as separate independent options look a bit suspicious to me. in libtorrent you can specify 1 or more (IP,port)-pairs and (interface,port)-pairs to listen on. i.e. if you specify an IP, the interface is implied (whichever interface the IP is configured of) or if you specify an interface, all IPs configured for that interface are used.

arvidn commented 4 years ago

(also, from libtorrent's point of view, there's no need to restart when changing the listen interface/IP)

Aerozolic commented 4 years ago

For me if I leave it to Any interface and just set the IP address to VPN IP then it still doesn't work. Shows Connection Status: Offline

arvidn commented 4 years ago

are there any configurations that do work with the VPN?

ghost commented 4 years ago

are there any configurations that do work with the VPN?

All interface option works. Meaning listening on 0.0.0.0 probably.

Aerozolic commented 4 years ago

are there any configurations that do work with the VPN?

All interface option works. Meaning listening on 0.0.0.0 probably.

For me selecting Any interface works as much as it's not showing offline but there's something wrong with ports because I get the "yellow icon" and DHT: Nodes continue to be 0 plus canyouseeme.org says that the specified port is refusing connections while the port is forwarded and is said to be open if I use some other bittorrent client.

ghost commented 4 years ago

are there any configurations that do work with the VPN?

All interface option works. Meaning listening on 0.0.0.0 probably.

For me selecting Any interface works as much as it's not showing offline but there's something wrong with ports because I get the "yellow icon" and DHT: Nodes continue to be 0 plus canyouseeme.org says that the specified port is refusing connections while the port is forwarded and is said to be open if I use some other bittorrent client.

I know another guy who had same issue. His ports were forwarded but he wasn’t getting any sort of incoming connections and his canyouseeme was returing error as well. It got fixed after he downgraded to 4.2.1 (libtorrent 1.2.3). Probably the listen sockets are not being opened correctly.

arvidn commented 4 years ago

@Aerozolic does your VPN provider assign you a public IP address? My impression is that it's common for VPN providers to not map ports to pass through incoming connections.

Can you make sure that the indications that you are connectable, while using the VPN, aren't trying to connect to you from your local internet connection? (i.e. not through the VPN). I'm suggesting that you not being connectable may be a sign of the VPN working as it should (and that you being connectible be a sign of the VPN being circumvented).

Aerozolic commented 4 years ago

@Aerozolic does your VPN provider assign you a public IP address? My impression is that it's common for VPN providers to not map ports to pass through incoming connections.

Can you make sure that the indications that you are connectable, while using the VPN, aren't trying to connect to you from your local internet connection? (i.e. not through the VPN). I'm suggesting that you not being connectable may be a sign of the VPN working as it should (and that you being connectible be a sign of the VPN being circumvented).

Before the latest qbittorrent update everything was working fine so this can't be a configuration issue. I had tun interface selected in qbittorrent and the connection icon was green. Nothing has changed aside from the app update. If it was a configuration issue then other apps would report that the port is closed as well. And I have selected "Send all traffic over VPN connection" in my OpenVPN client. If I disable VPN then the connection icon turns green and DHT nodes start to increase.

Edit: Qbittorrent also shows the same thing for all trackers: "Not Working"

arvidn commented 4 years ago

Before the latest qbittorrent update everything was working fine so this can't be a configuration issue.

And you're confident it didn't just look fine but was circumventing the VPN, right? I'm suggesting (not claiming, because I don't know) that the fact that it was "working" before may have been because of a bug, that has now been fixed.

@FranciscoPombal (or any other qbt developer), it would be helpful to know what the libtorrent listen_interfaces setting is set to.

arvidn commented 4 years ago

trackers not working when using a VPN sounds in line with the title of this ticket. I just wish I could reproduce it. Does anyone experience this on PrivateInternetAccess or NordVPN? (those are the ones I've been testing with), on linux.

ghost commented 4 years ago

@sledgehammer999

Aerozolic commented 4 years ago

Before the latest qbittorrent update everything was working fine so this can't be a configuration issue.

And you're confident it didn't just look fine but was circumventing the VPN, right?

I'm suggesting (not claiming, because I don't know) that the fact that it was "working" before may have been because of a bug, that has now been fixed.

@FranciscoPombal (or any other qbt developer), it would be helpful to know what the libtorrent listen_interfaces setting is set to.

Just tested it on my other computer that has the same VPN provider (Torguard) and forwarded ports, same torrent, same qbittorent configration but version 4.2.1. Connection icon is green, DHT nodes show more than 100 and trackers are working (not all ofcourse since it’s a torrent from public site).

gothicserpent commented 4 years ago

Just wanted to add: I have confirmed my mullvad VPN does issue a public IP which is different from my default IP. I tested this by downloading a magnet link from my mullvad vpn's web site which tests to see if my original IP is being leaked, and the site confirms it is not being leaked in either 4.2.1 or 4.2.2. here's the site i used for that from my vpn provider: https://am.i.mullvad.net/torrent (this is similar to using sites from the google result of torrent leak which lists sites that test if your torrent client is leaking your original ip)

the settings: --mixed_mode_algorithm=prefer_tcp , --max_pex_peers=0 --anonymous_mode=1, and outgoing_interfaces are all things that may have a role to play here. If it's possible to use NordVPN with OpenVPN as a test, this might help as a testing benchmark.

Finally, another series of users reported something similar at issue #12297 where they cannot use their VPN with 4.2.2

Aerozolic commented 4 years ago

I got some improvements. If I switched the OpenVPN protocol from UDP to TCP then DHT nodes started to increase and trackers changed from not working to working. But there's still an issue with connectivity. Canyouseeme now gives "Connection timed out" instead of connection refused. Maybe this info helps, maybe not.

FranciscoPombal commented 4 years ago

@arvidn

@FranciscoPombal how do those settings translate to the libtorrent listen_interfaces setting?

The fact that they are presented as separate independent options look a bit suspicious to me. in libtorrent you can specify 1 or more (IP,port)-pairs and (interface,port)-pairs to listen on. i.e. if you specify an IP, the interface is implied (whichever interface the IP is configured of) or if you specify an interface, all IPs configured for that interface are used.

are there any configurations that do work with the VPN?

@FranciscoPombal (or any other qbt developer), it would be helpful to know what the libtorrent listen_interfaces setting is set to.

I must say I am not too familiar with that code, there could be something wrong with it. @sledgehammer999 might be able to help you more with that right now, as he has worked on that code most recently.

sledgehammer999 commented 4 years ago

(or any other qbt developer), it would be helpful to know what the libtorrent listen_interfaces setting is set to.

@arvidn The exact string is printed in the log.

@ everyone who has the problem. Report the log string for @arvidn to see. In english it is logged this way:

Trying to listen on: %1

where %1 is the string passed to listen_interfaces

FranciscoPombal commented 4 years ago

In addition:

gothicserpent commented 4 years ago

here is my output - https://pastebin.com/Fujhqbqd

ghost commented 4 years ago

trackers not working when using a VPN sounds in line with the title of this ticket. I just wish I could reproduce it. Does anyone experience this on PrivateInternetAccess or NordVPN? (those are the ones I've been testing with), on linux.

Trackers are working fine here. Just that we're not being able to connect with BT peers. uTP works. The title is appropriate.

I just tested with NordVPN as well and was able to reproduce the exact same issue. This is not a specific provider issue. All major VPNs are affected more or less. @arvidn

Aerozolic commented 4 years ago

In addition:

  • If on Windows, open CMD, run route print and post the output

  • If on Linux post the output of ip route

And on Mac? Same as Linux?

FranciscoPombal commented 4 years ago

@Aerozolic

And on Mac? Same as Linux?

I googled a bit and found on mac you should use netstat -nr instead.

FranciscoPombal commented 4 years ago

Note for @arvidn: In qBittorrent's logs, the lines that read ... Successfully listening on IP: ... are basically the contents of a listen_succeeded_alert (endpoing.address(), sock_type and endpoint.port()).

Full detail here: https://github.com/qbittorrent/qBittorrent/blob/82c23e67a47c79983a128338d338f055c9a054dc/src/base/bittorrent/session.cpp#L4698

Aerozolic commented 4 years ago

I feel super dumb right now because I was able to fix all my issues with one simple step - restarting my computer. The error that I previously got was: Failed to listen on IP: xx.xx.xx.xx, port: UDP/xxxxx. Reason: Address already in use. So I contacted my VPN provider and they said that it’s not an issue with the VPN server and that the client’s IP address is in use by some other service. Since I didn’t have any other ideas I just restarted my computer and qbittorrent started to work again. Connection status green while VPN interface set.

ghost commented 4 years ago

Just to add, it looks like the network interface selection is completely broken at the time.

If you bind to VPN interface and for some reason your VPN disconnects/crashes and has no killswitch then your IP will get leaked anyway because the interface somehow switches back to a working one even when the VPN interface is selected. qBt's interface selection option makes it look like it's not gonna use any other interface when there's no internet. But in reality it's not honouring that.

I think this thing requires a rework as such IP leaking can be potentially fatal unless user is using a killswitch. @FranciscoPombal

CeruleanSky commented 4 years ago

@an0n666 I noticed that also and commented in a different issue. Where just like you, if the vpn drops or disconnects, even if you select a specific interface, the program shifts to listening to every interface. Since VPNs often give different intranet ip address per connection, the bind to ip address option isn't as useful and has to be set to "any ip address", so it can't really solve this.

Here is an image showing this where I disconnect the vpn to simulate a dropped connection/crash/disconnect with qbittorrent 4.2.2 built with qt 4.13.2:

image

IP leaking can be potentially fatal unless user is using a killswitch.

Killswitches for many vpn software run scripts using the "net" and related commands which will likely be slower than qbittorrent, leaving a window open as qbittorrent shifts adapters.

FranciscoPombal commented 4 years ago

Just to add, it looks like the network interface selection is completely broken at the time.

If you bind to VPN interface and for some reason your VPN disconnects/crashes and has no killswitch then your IP will get leaked anyway because the interface somehow switches back to a working one even when the VPN interface is selected. qBt's interface selection option makes it look like it's not gonna use any other interface when there's no internet. But in reality it's not honouring that.

I think this thing requires a rework as such IP leaking can be potentially fatal unless user is using a killswitch. @FranciscoPombal

@an0n666 I generally agree, but just to be clear, I think the correct behavior is: If a specific interface is selected and it stops being available at some point, qBittorrent should stop working until a working one is manually selected.

If this is not the case (i.e. "Any interface" is selected), qBIttorrent should use whatever interface it wants, but should strive to only use one interface at a time for everything.

@arvidn does libtorrent provide any way to detect whether an interface, while available, might not actually allow any connections? For example some kind of dht not working alert that allows deducing that the interface leads nowhere.