search

qbittorrent / qBittorrent

qBittorrent BitTorrent client
https://www.qbittorrent.org
Other
26.35k stars 3.85k forks source link

Different torrents share progress for same file/pieces #11176

Open DougBarton opened 4 years ago

DougBarton commented 4 years ago

I love qBittorrent, and have been using it for years, but every once in a while I like to see what's out there. I tried Vuze recently, and while it has enough things I didn't like to rule it out for day to day use, one feature it had that I really liked was the ability to recognize when more than one torrent was trying to download the same file (I believe by checksum, as opposed to file name) and to share that progress between the torrents. It may also have been able to identify the same pieces across torrents by checksum, which would actually be more valuable. I also may be totally misunderstanding how it works, I am not an expert on bittorrent internals. :) I believe the features is called swarm discovery.

I like to download some older stuff with very few seeders, so this is a feature that would be very valuable for me. In addition to personal benefit for the individual qBittorrent users, it would also be good for the swarm. I've often found that by doing this technique by hand across multiple torrents it allows people to pick up pieces that they are missing, and encourages folks to stay on longer to share the pieces that they have. I've been able to complete many downloads of older torrents using this method.

This feature would be valuable enough to me that I'd be willing to put up a bounty for its completion.

coolio2013 commented 4 years ago

+1 Just thought about that today again, because I have 2 different torrents, with same content, same filesizes, but different hashes. And I can't complete any of them (there are no seeds). Then I found I even can't add a peer manually (see #11180).

I guess this is technically difficult (perhaps piece sizes, or filenames) and probably can work only based on filesizes. It has been discussed somewhere, I think there also was a proposal for an addition to the BT protocol, but can't find that currently.

Because of this feature I tried to install Vuze/BiglyBT (failed due to Java PITA). It seems it is labelled 'swarm merging' over there (https://github.com/BiglySoftware/BiglyBT/wiki/Swarm-Merging).

xavier2k6 commented 4 years ago

swarm merging is a libtorrent/bittorrent v2 protocol thing I believe @arvidn @ssiloti may be able to give better insight to this.

DougBarton commented 4 years ago

Nice to see that there is interest. :)

arvidn commented 4 years ago

as far as I understand, this could be an offline tool, separate from the torrent client. But I wonder, has anyone measured, and quantified, how often this happens?

DougBarton commented 4 years ago

I'm not sure why an off-line tool would be better than building it into qbittorrent.

As for frequency, I often go hunting after old stuff, so it's common for me to have four or five sets of torrents where each set is actually one file (or one set of files). It gets very tedious to periodically stop the ones lagging behind in order to copy over the latest version from the particular torrent that pulled ahead that day.

arvidn commented 4 years ago

I'm not sure why an off-line tool would be better than building it into qbittorrent.

because it would be simpler to build, and there's a shortage of software engineers in the world.

gitthehubs commented 4 years ago

Same wishlist here.. swarm merging :) https://wiki.vuze.com/w/Swarm_Merging

Vuze source code is available on Sourceforge.net.. but i have no idea if you can use that for implementing in qbittorrent, but you can at least look in the code how they did it :) : https://sourceforge.net/projects/azureus/

eadmaster commented 3 months ago

Possible duplicate : https://github.com/qbittorrent/qBittorrent/issues/9416