Previously downloaded files are moved into unwanted folder

[OlegYch]

Say i have a torrent with files a,b,c I previously downloaded the file c. Then i added the same torrent but now it contains files a,b,c,d. I select only file d for download as i already have file c and don't want anything but d to be downloaded. qbittorrent moves the c file into unwanted folder, while a reasonable thing to do would be not touching existing files.

[disfated]

+1

[sledgehammer999]

This is expected behavior. This behavior expects that you don't do things like that. You can't point a torrent to a folder with preexisting torrent files and expect it to not touch them. The torrent might need to recheck them, move them, rename them, continue downloading them and delete them if you tell it to "also delete the files on the hard disk". There is no way that a client can tell when you want to touch the files and when you don't. If they exist in the save path the client assumes that it can manipulate them.

I'll close this, but you are free to continue commenting and maybe propose possible solutions. Thank you.

[OlegYch]

On Sunday, September 1, 2013 10:48:21 PM, sledgehammer999 wrote:

This is expected behavior. This behavior expects that you don't do things like that. You can't point a torrent to a folder with preexisting torrent files and expect it to not touch them. The torrent might need to recheck them, move them, rename them, continue downloading them and delete them if you tell it to "also delete the files on the hard disk". There is no way that a client can tell when you want to touch the files and when you don't. If they exist in the save path the client assumes that it can manipulate them.

I'll close this, but you are free to continue commenting and maybe propose possible solutions. Thank you.

— Reply to this email directly or view it on GitHub https://github.com/qbittorrent/qBittorrent/issues/871#issuecomment-23631715.

okay, moving back to utorrent then

[disfated]

It is obviously NOT expected behaviour, because "Don't download" doesn't mean "Remove" or "Move away" - it only means "Don't download"... utorrent, deluge, transmittion handle it the right way. The last two are open source...

[sledgehammer999]

is obviously NOT expected behaviour, because "Don't download" doesn't mean "Remove" or "Move away" - it only means "Don't download"... utorrent, deluge, transmittion handle it the right way. The last two are open source...

Correct me if I am wrong but those treat unwanted+wanted files the same, they don't move unwanted files in a different directory, they let them sit on the same directory as the wanted files.

Also what would happen on those 3 if you did this:

1. Have a torrent with files a,b,c and download b.
2. Remove torrent but leave data
3. Add torrent with files a,b,c,d and choose to download d
4. Remove torrent and choose to delete data too.

What happens then at 4? Do they also remove c?

[disfated]

Do they also remove c?

Yes. When your add 2nd torrent-file, client is usually forced to rehash data and if finds file b valid it marks it as downloaded (just if it was downloaded via 2nd torrent-file). From this point file b is no way to distinguish which way it has previously been downloaded, and is recognized the same as all the other files. When user chooses to delete files - it should be deleted, as all other files.

those treat unwanted+wanted files the same

uTorrent has it. If some file ever been chosen by user as downloadable, client allocates target file and never deletes it (if not explicitly asked DELETE, or data collisions found). All unwanted data (pieces of not downloadable files) is stored in private special-format file (minimal size) that lies in torrent directory. When user deletes torrent (not data), utorrent only deletes this special file, and all other structure remain clean and no wanted files being deleted. Note, that if downloadable file is checked as not downloadable, client doesn't move it's data to the private storage file - i.e. it remains "wanted" although is not currently marked as downloadable. That actually means "ever wanted files remain wanted forever".

Deluge, Transmission don't actually have unwanted / wanted. All pieces of "unwanted" files are stored in fully allocated target files, which is not efficient becauses (1) it steals free space, (2) no way to differ unwanted fragment-files from wanted partially downloaded files, so when user deletes torrent (not data), they are left as is (simply nothing is deleted), and directory structure may be completely littered.

[sledgehammer999]

I have an appropriate counter-response to your argument but I won't say it. I am currently contemplating with the idea of removing the .unwanted feature altogether. I was thinking that although this feature is nice, it is breaking the de facto standatd practice of bt clients of downloading everything to the save path. I remember that in the old days, one of the "awesome" feaures of bittorrent protocol that separated it from other p2p programs was that you could use one client to start a download, close it and choose another one to continue the download(after rechecking). So I feel like qbt is breaking that convention. It's something like bitcomet's stupid padfiles thing. What do you think?

[disfated]

I believe there's currently no ideal solution for all cases. As to me, uTorrent's behaviour is most trade-off. Removing .unwanted? But we still have to store "padfiles" somewhere. If you store them in target paths - you waste space and pollute directory structure. If you use some .padfiles.qbt (only actually downloaded data stored - no full allocation) - you breaking (some of) portability (as you sad, feature when any client generates data structure valid for all other clients)... So, choosing between ton of gigabytes of disk space completely lost and theoretical portability of downloaded data - you could expect most users to choose their gigabytes :) The ideal solution, to my mind, would be to standardize .padfiles format, so that all clients could use it, keeping data sturcture thrifty, clean and portable.

[disfated]

Also, as many of torrent-clients are open-sourced, i believe that developing and implementing "padfiles" format wouldn't take much resistance as we can see in other software areas...

[Belove0]

What do you think?

The .unwanted folder has always been undesired from my perspective; It's hidden from the user, it's non-standard, it's undisclosed to the average user, and the user can't manage those files. If their file system/settings don't allow sparse allocation (and possibly pre-allocate files has an impact) these files take up unnecessary space on people HDs, not easily traced for the typical user.

Beyond that, I never delete files using my torrent client, so I was unaware of the differences between clients there, but I do agree to one pro for keeping files in same directory: inter-client compatibility, as you mentioned. uTorrent breaks client compatibility in a minor way with it's partfile format, but I LOVE partfile. Typically only a small amount of data is there anyway, so another client won't be missing much. It keep the directory clear of essentially corrupt files that the user isn't interested in that are only needed for technical reasons. uTorrent does not move existing files to the partfile, though. If an existing partial file exists, it will download new pieces to it. Also, if a file is re-selected for download in uTorrent, it will create the file and move the pieces from the partfile to the proper file name. (if the file already exists, I don't know what it does with the chunks in the partfile, but it certainly doesn't delete the existing file by that name, and does start downloading to it -- so it may well overwrite as necessary from it's partfile since that is conceptually in line with writing over the file at all).

Aside on inter-client compatibility: I use the feature for a temporary extension for partly downloaded files (e.g. .!qb) in all clients regardless. I still associate that extension with my media player, but I can easily identify which files are completed from the file system. It would be really nice if all clients used the same extension for that feature, but no biggie.

remember that in the old days, one of the "awesome" feaures of bittorrent protocol that separated it from other p2p programs was that you could use one client to start a download, close it and choose another one to continue the download(after rechecking). So I feel like qbt is breaking that convention. It's something like bitcomet's stupid padfiles thing.

Yes. Another awesome feature of BT and a problem highlighted by this feature request that I was not aware of directly impacts the way I work with torrents. I realized that .unwanted might interfere with this procedure, but I did not realize that qBT would move files to the .unwanted that didn't start there. That makes a usual practice of mine not really workable in qBT: When running across torrents that are either slow or have no full seeders (but many complete/mostly complete files), I often recover the torrent by finding the correct files in multiple torrents or other sources online. I download them all to the same directory (which I also do with unrelated torrents as well if the files have a relationship to me). I can even download files from multiple torrents simultaneously being wary of the impacts it will have on edge pieces, sometimes having to avoid simultaneous downloads in some clients (prior experience: uTorrent) of files that share pieces in any of the selected torrents, because a file sharing conflict may occur. (clients could avoid this error by retrying or not locking for writes, and/or locking only the bytes they are writing to rather than the whole file if they expected the behavior). >>>>> I select only some files from each torrent to speed download or to enable the recovery of a full seed of a "dead" torrent. When I have all the files for my favored torrent(s) in the directory, I select all files for download and then recheck them all. Voila, full seeds, and I have all the files as quickly as possible for myself as well.

P.S. My above described use case I believe falls within the scope of BEP.38, which is intended to share data between torrents as well as other files the user has. I commented on a libtorrent feature request previously for BEP.38 support: https://code.google.com/p/libtorrent/issues/detail?id=447 To my knowledge, no one is implementing BEP.38 yet.

• I am worried that if I try my method to share data between torrents in qBT, after reading this bug, though, that I will end up with problem when I select files and .unwanted files are moved (depending on the behavior when qBT discovers the files already exist), or that qBT will move my downloaded/downloading files into .unwanted. The deletion issue could be a problem as well, but since I rarely if ever delete files other than the .torrents from within a BT client, it doesn't effect me much.
• My vote: emulate uTorrent. Duplicate their partfile format (if possible) for maximum compatibility. Other than the file sharing issue possible with my usage which in uTorrent automatically pauses the torrent in an error state, uTorrent's behavior is my ideal.

Alleviate the file sharing issue for hacks like me and I'll be very happy until a thorough BEP.38 implementation turns up. Which may be a while as the spec is very broadly defined and would require a lot of planning to decide how to implement it (and maybe additional BEPs for parts of the design best shared between clients). Also in re. qBT/libtorrent, there are outstanding issues with file access and queuing that I think any substantial implementation of BEP.38 would likely exacerbate.