[Question] Is it expected that global upload speed also limits download speed?

[levicki]

I see that this was asked already in #12192, but please bear with me for a while because I think the answer isn't as black and white as presented in that previous issue by @FranciscoPombal.

I have noticed that when global upload limit is set to say 64KB/sec, the download is very slow (say ~1MB/sec). Once I remove the global upload limit download speed maxes out at 30MB/sec regardless of whether I am actually uploading something to peers or not.

To me this looks like TCP ACK packets are counted towards the global upload limit. Can someone please confirm this?

If so, that means you can't really set an actual torrent upload speed limit for seeding without affecting your download speed.

I just checked on my router queue and maintaining 30MB/sec download speed without uploading to peers (it's a well seeded torrent so nobody is downloading from me and I am not getting choked) seems to require between 1.5 and 3Mbps upload just for the TCP ACK packets.

That means that limiting global upload speed in qBitTorrent to say 1Mbps will severely throttle your download speed.

In my opinion, there should be a way to set actual torrent upload speed limit separately from global upload speed. Setting global upload speed to ~95% of your upload capacity is good to make sure your WAN upload is never maxed out, but it should also be possible to limit actual torrent upload speed for people who have highly asymmetrical download and upload ratio (say more than 10 : 1) and those who have more heavy Internet users in the household and don't want qBitTorrent to use all the available upload capacity.

I intentionally phrased this as a question, not a feature request or bug report, because I am interested in your view on this. We can easily change the issue title if necessary.

[zecanard]

Have you tried unchecking “Apply rate limit to transport overhead”?

[Rootax]

Oh, a Mikrotik user :)

I agree with you, I saw the same behaviour.

More over I believe high speed uploader are "rewarded", with the bt protocol, no ?

[levicki]

Have you tried unchecking “Apply rate limit to transport overhead”? I must admit that I am not sure, I will have to check the state of that checbox.

As a non-native English speaker I assumed transport to mean BitTorrent transport in general, not TCP transport specific overhead.

[FranciscoPombal]

@jdsys271 In the case of a traditional BitTorrent connection, which sits on top of TCP, there is some overhead associated with sending ACK packets mostly. That is to say, in the process of receiving data, you necessarily have to transmit a tiny bit as well, due to the way the transport protocol itself works.

So, for example, if you are downloading at 1 Gbit/s with a TCP-based protocol you might see 10 Mbit/s in uplink usage more or less (I think, actual value might be slightly off). This is called the transport overhead.

"Applying rate limit to transport overhead" simply means that whatever the transport overhead is, it will count towards your rate limit. So, for example, if you have a 20 Mbit/s rate limit and protocol overhead is taking up 10 Mbit/s, your maximum throughput for actual upload to peers will be 10 Mbit/s. If this option were disabled, you would be able to upload to peers at the full 20 Mbit/s, but you would actually be using 30 Mbit/s of your connection's uplink. Of course, if this option is enabled and you set the rate limit too low, it will actually prevent you from downloading at faster speeds.

I intentionally framed this explanation as "how uplink overhead influences payload download" for simplicity, but the same is true in reverse as well; to upload data, you necessarily have to dowload some data, which is transport overhead as well (in the case of TCP, the ACK packets the other peer sends you, for example).

[wefalltomorrow]

Is it expected? No, but qBittorrent (and uTorrent) intentionally does this. Other clients such as Tixati and Deluge do not do this, I assume it's to deter people from slow seeding/leeching.

[FranciscoPombal]

@wefalltomorrow It's not to "deter people from slow seeding/leeching.", it's to prevent people from complaining about "qBittorrent is not respecting my rate limits!". It's better to have it on by default.

[levicki]

@FranciscoPombal I agree it is better to have it on by default, but perhaps there is a possibility to explain it better to people with less understanding of how networking protocols work?

I think it might be better if this wasn't a checkbox. I think that providing two separate upload limits would better convey the difference to the users and allow them to have better control. For example:

1. Total upload rate [____] (your internet connection upload capacity, includes TCP protocol overhead)
2. Torrent upload rate [____] (must be less than total upload rate)

Not sure if it is techincally feasible for implementation though.

Providing separate upload rate limits would allow users to specify maximum upload rate which should never be surpassed (i.e. their total ISP link upload capacity which is something they either know or can find out), and the rate at which they want to upload/seed.

With the current system that doesn't seem possible -- you can either have total upload rate match your maximum upload bandwidth (which doesn't allow you to limit seeding speed at all), or if you uncheck the protocol overhead checkbox you can have limited seeding speed, but combined with protocol overhead this can still saturate your maximum upload bandwidth to the point of making your internet connection useless for anything (or anyone) else in your household.

True, you could try to eyeball the upload rate that lets you download at full speed and still leaves some upload bandwidth for other use (or users), but protocol overhead is not something regular users should be expected to understand and it's rather hard to calculate/measure even for those who know about it (it depends on whether plain or selective ACKs are used, on your MTU/MSS, on TCP window scaling, and whatnot), so aside from mentioning it as part of total upload rate it is not very useful as a user-facing option since they cannot be aware of how much upload their download might be generating.

[FranciscoPombal]

@levicki I think this is a solution in search of a problem.

• To improve the UX, perhaps tooltips could be added to those checkboxes (example: https://github.com/qbittorrent/qBittorrent/pull/12027) with a quick and understandable explanation of what they do.
• You can always limit seeding speed. Checking the box to apply the rate limit to the transport overhead simply means that, in practice, the seeding speed limit will be slightly less than the overall upload limit you set. This is not a problem IMO, unless the user's connection is severely imbalanced (like 1 Gbps down/10 Mbps up or less), which doesn't happen in practice (at least I sure hope not), and in that case, the bottleneck is the user's connection itself anyway. For "reasonable" setups, where up/down speed are reasonably matched and the network hardware can handle it, the uplink transport overhead will be a relatively small percentage.

[levicki]

@FranciscoPombal

Let's say I have 62.5 MB/sec download and 6.25 MB/sec upload, which roughly corresponds to 500 / 50 Mbps (so a DL / UL ratio of 10 : 1, which is common, but in practice there are still cases where you can have up to 20 : 1 such as on cable, xDSL, or satellite links).

Let's say that I want to limit my upload to 1 MB/sec out of those 6.25 MB/sec, but still be able to download at 62.5 MB/sec.

Let's also assume the following for the sake of this example simplicity:

• My MTU is 1500 and useful data is 1472 bytes per packet
• All ACK packets are 40 bytes
• There is no window scaling
• There are no packet reorders and retransmissions

To download at 62.5 MB/sec I need to receive 44,522 packets, and send that many ACK packets which means I must send ACKs at 1.7 MB/sec. If I capped my total upload speed including protocol overhead to 1 MB/sec that means that even without seeding my maximum download rate will be capped at 36.7 MB/sec.

Now of course, the real calculation is much more complex and the number of ACKs needed will be probably lower because of window scaling, but it may also go the other way (retransmissions due to congestion, fake peers corrupting swarm) and there are also going to be other control packets (SYN for creating new connections, all the other traffic for updating trackers and finding peers, asking peers for pieces, etc).

What I have actually observed in practice by packet marking and monitoring on the router is that for downloading at 30 MB/sec I am uploading 0.75 MB/sec worth of ACKs.

If we extrapolate from that using simple proportions, then I would be sending 1.5625 MB/sec for 62.5 MB/sec download which means that my download speed would indeed be capped if I set the 1 MB/sec upload limit -- my capped download rate would be 40 MB/sec, which is pretty close to the approximation I just calculated above.

That means I have already reduced my download capacity to ~2/3 without ever limiting actual seeding rate, and if I am seeding at the same time, then my download will crawl to a stop which will further discourage me from seeding unless I am tech-savvy enough to be aware of all this and take it into account when setting the upload limit with overhead, which I am sure you will agree is something most users won't be capable of and/or willing to do.

TL;DR -- I still think those limits should be separate, but I will settle for clearer labels and/or tooltips if you really don't think what I am bringing up has any merit.

[FranciscoPombal]

@levicki Thanks for the detailed post, I now better understand your reasoning behind splitting the upload limit controls in the previous post. But, I still think it is not the right solution; it's just more room for a user to shoot themselves in the foot. Building on your 500/50 connection example, if there is no such overhead checkbox and the user set:

1. Total upload rate: 1 MB/s (this is presumably a limit for what qBittorrent is allowed to use in total, it would not make much sense to make this the same as the connection's total capacity)
2. Torrent upload rate: 0.5 MB/s

this would lead to only 0.5 MB/s left for overhead, which would again cut down on the download speed potential. On the other hand, if the user set "Total upload rate" to their connection capacity, they would be surprised that qBittorrent would consume much more upload traffic than anticipated (if they were downloading at full tilt and seeding up to the limit as well, in this example qBittorrent would be generating 0.5 (seeding) MB/s + 1.5 (overhead) MB/s = 2 MB/s upload), precisely because a regular user does not understand what transport protocol overhead is.

Currently, the checkbox to apply the limit to transport overhead defaults to on. Thus, if the user sets upload limits, they can be confident that qBittorrent will never be uploading at a rate greater than the value they set, even if they don't know how much of it is transport overhead and how much of it is actual seeding. I think there is a reasonable expectation that a user understands that in BitTorrent, the more you upload, the more you can download, and the less limits you put in your connection, the better the experience is.

If the user complains about slow speeds while using limits, the solution is quite simple. Either increase the upload limit, or uncheck the checkbox, at the cost of additionally uploading "a bit more than what the limit says" (the user can then use some kind of monitoring utility to get an idea of what this "a bit more" is determine if it is acceptable or not). Alternatively, if the user wants both a low upload limit and to prioritize seeding, cap the download speed to keep the upload overhead manageable, so that most of the upload limit is used for actual seeding.

Perhaps the FAQ or the wiki could have an entry explaining this in simple terms. Something like "Do you use upload limits and experience slow download and/or slow seeding speeds? In order to download at the max your connection speed allows, your upload limit should be no lower than 2.5%~3% of your connection's download speed. Everything above that is guaranteed to be used for seeding, now matter how fast you are downloading. If you want an even lower upload speed limit without compromising seeding speed, you can limit the download instead. Alternatively, uncheck Apply rate limit to transport overhead, which guarantees all upload within the limit is dedicated to seeding, but expect additional upload behind the scenes, depending on how fast you are downloading."

[levicki]

@FranciscoPombal

Currently, the checkbox to apply the limit to transport overhead defaults to on. Thus, if the user sets upload limits, they can be confident that qBittorrent will never be uploading at a rate greater than the value they set

Currently if they set Total Upload Rate to 2 MB/sec with checkbox enabled that includes overhead.

If controls were separate as I suggested, and they set only Total Upload Rate to 2 MB/sec while leaving Torrent Upload Rate at 0, they would get identical behavior as they get now.

Therefore, default behavior would not (and does not have to) change by introducing a separate control.

even if they don't know how much of it is transport overhead and how much of it is actual seeding

IMO that's part of the problem we are trying to solve, because we have this equation:

In the current implementation, both TorrentSeedingRate and ProtocolOverhead are unknown variables so it isn't solvable, and frankly I fail to see how reducing the number of unknown variables in an equation is somehow a bad thing for the user.

I agree that this is a hard problem to solve, and certainly there might be better options available, but I also hope they include user having more control because in that case an option of Auto could be added as well while now it can't.

[FranciscoPombal]

@levicki

The thing is, the "average worst-case" protocol overhead is known, it's about 2.5%~3% of the connection's max download rate. So the equation only really has one variable, the torrent seeding rate, which is whatever the user wants. If the user wants to further limit the total upload rate, while keeping the torrent seeding rate, they must necessarily limit the download speed as well. And it's even rather easy to figure out but how much exactly you need to limit the download speed to achieve this. Perhaps it would come in handy to have an online calculator for this, in the spirit of http://infinite-source.de/az/az-calc.html.

Currently if they set Total Upload Rate to 2 MB/sec with checkbox enabled that includes overhead.

If controls were separate as I suggested, and they set only Total Upload Rate to 2 MB/sec while leaving Torrent Upload Rate at 0, they would get identical behavior as they get now.

No doubt it is possible to achieve the same thing as now, but differently. I think it's more confusing that way though.

What if Torrent Upload Rate was a minimum guarantee, instead of a limit? Perhaps with a different name even, like Reserved for seeding? That way, if the user sets:

• Total upload rate limit: 2 MB/S
• Reserved for seeding: 500 KB/s then qBittorrent will let the overhead be 2 MB/s max, but as soon as something starts seeding, it will be allowed to eat into the overhead's share of the limit, up to 500KB/s. This way, download speed is maximized whenever there is a chance, but upload is always prioritized, even if it impacts download speed. I think this what the majority of users want.

[levicki]

@FranciscoPombal I agree that maybe different wording would help, but I wouldn't assume that all users want to prioritize seeding over downloading all the time. I personally prefer to have choices, that's why I was suggesting more options.

People are so eager to criticize Apple for their "one size fits all" approach to both hardware and UI/UX design and for their "we know best what users want" stance, but on the other hand it seems that a lot of individual developers easily fall into the same trap, even though they would probably criticize others for doing the exact same thing.

You can close this issue if you want, whatever improvement you implement to clarify the current situation will be welcome.

[FranciscoPombal]

@levicki

You think the alternative I proposed is more restrictive? I believe it achieves the same as yours above (https://github.com/qbittorrent/qBittorrent/issues/12491#issuecomment-613980955) but with slightly different (better) semantics and wording:

What if Torrent Upload Rate was a minimum guarantee, instead of a limit? Perhaps with a different name even, like Reserved for seeding? That way, if the user sets:

* Total upload rate limit: 2 MB/S

* Reserved for seeding: 500 KB/s

then qBittorrent will let the overhead be 2 MB/s max, but as soon as something starts seeding, it will be allowed to eat into the overhead's share of the limit, up to 500KB/s. This way, download speed is maximized whenever there is a chance, but upload is always prioritized, even if it impacts download speed. I think this what the majority of users want.

I don't think this only requires a wording change though, I think the implementation could be more complex than that.

I wouldn't assume that all users want to prioritize seeding over downloading all the time.

I guess there could additionally be a button on the main UI (similar to those that enable/disable global limits) that enabled/disabled the "Reserved for seeding" quota.