Open anonymous82 opened 6 years ago
qbittorrent / Tools / Options / Advanced there is option "Recheck on completion", maybe it can work around that? Btw, it looks similar to this: https://github.com/qbittorrent/qBittorrent/issues/7850
I also encountered this issue today when downloading 6 torrents at a time. The first one completed like normal, and after starting the three next ones, I started two more that had a higher priority to me so I paused the 3 torrents started in the "second batch". The two torrents I started most recently finished like normal, while the 3 that I had paused all suffered from this issue when they "completed". I resumed downloading the 3 at the same time.
When looking at the status bar in the main torrent list, labeled "Done" in the header, the three affected torrents showed 100%. However, when inspecting the individual files in the "content" tab, the biggest file in all three torrents showed as 99.9% completed. Going back to the "general" tab, I observed that they were missing 5, 7 and 7 pieces out of roughly 4500 pieces (the total number of pieces was similar for all 3 torrents).
Upon forcing a recheck, all 3 torrents completed instantly.
I currently have 600 torrents in my list, with many more completed in the past and on other devices, and I have never encountered this issue before updating to 4.0.1.
Happends to me as well. Edit: Transmission with the Remote GUI addon has mostly the same feel and features as Qbittorrent, and doesn't have these ludicrous bugs that the devs are incabable of fixing even when they have been known for over a year.
Suggest you guys give it a go if you are getting as annoyed, as I did.
Started happening to me as well (just randomly) on 4.0.3. If I pause-resume-force resume etc it doesn't help. I have even done a recheck on one and it didn't finish, I had to exit and reopen qbit to get it finished. Also odd was the "run once complete" command I have didn't run when I reopened to make it finish. On each one overall status is 100% but one of the files will be 99.9% (with tons of seeders).
Edit: ok force recheck is working to restart them now. And they will "run once complete". Before it may not have worked because I was already downloading more then my que allowed.
Well after updating to latest version, I'm happy to say it doesn't really happen anymore.
Before this, forcing a recheck was the only way to get it complete as I mentioned previously.
This is still happening in 4.1.1.
Still happening on latest (4.1.1) x64 running on windows server 2012. Very annoying. Will point out that suggestions above to use the "recheck on completion" option won't work as the torrent is technically not complete. I'm mostly dealing with larger torrents (50-300GB) and every time I've checked one of these stalled torrents, there's anywhere from 1 - 50MB that hasn't downloaded. Rechecking these manually is very tedious, not to mention the unnecessary I/O (and TIME) it's been eating up.
Upgraded to 4.1.2 today and it doesn't seem to be happening anymore... But it could just be dumb luck. I'll keep y'all updated.
Happens on 4.1.2, been happening forever. Restarting qbittorrent completes them.
bugs for this date back 5+ years they wont be fixing it anytime soon. the only true fix is ditch qbt and use something that works properly
https://transmissionbt.com/ + https://sourceforge.net/projects/transgui/ is an alternative with a familiar feel to it.
Yep I get this exact problem as well and more annoyingly I now have the RSS re-downloading the same torrents (only certain ones) OVER AND OVER AND OVER AND OVER and they just keep coming back as unread even though they were downloaded/deleted a week ago
Still an issue. The more I use qbt, the more half a decade old issues I find but the alternatives are even worse. Force recheck solves is, I'm guessing it's a caching issue of some sort.
Still having this issue as well
In recent times I only got a stall at 99% when I ran out of space. After I resolve it the torrent is guaranteed to get stuck at 99%
Still happening just now. I'm going to use something else unfortunately.
For all would be Googler's just select all the affect torrents, and Force Recheck.
Happens to me as well in 4.1.5. Stalled on 100%, disk usage is also 100%. Seems to be happening randomly, or I did not find any pattern.
4.1.6 on Windows, still occuring.
I have "Recheck on completion" enabled.
I love how this has been reported and re-reported for years without so much as an acknowledgement from the developers. First class, no, really.
As in last comment, this still happens in 4.1.6.
Shout out to all 5.x users with this issue from the future. Is that twat Trump still in charge?
Maintaining open source software is shit work, and soul-croushing after the honeymoon period. If you don't like it submit a patch or move to a different software. Don't insult the creators.
In 4.1.7 it happens much more frequently that it was in 4.1.5 (4.1.6 was too buggy to use it)
Still happening to me on 4.20, even with "Recheck on completion" turned on. Quitting and re-opening the client doesn't move them past stalled (100%).
Happen for me with 4.2.1. But, in my case, I think I know what was the source of the problem. I had a lot of downloads at the same time and my hard drive was not able to keep up with downloading/validating pieces (and other tasks I was running on the same drive). The torrents were stalled at 100% done, but there were green parts in the "General \ Progress" bar (i.e. partial pieces). I "paused" all the others torrents and qbittorrent was able to validate the 'green pieces' and complete the download. There might be a bug where a '100% downloaded but partially validated' torrent is too low in the priority list.
happened to me as well. it usually doesn't, but yesterday i downloaded a lot of big torrents at the same time (which is a very rare occurrence) and a few torrents got stalled at 100% overnight. a manual force recheck fixed them.
tsbus - Same for me, Its been a ongoing issue for me for a long time on multiple computers/versions but its rare except when downloading allot at once. If I download 100 torrents 1 at a time it might not happen at all, if I download 100 - 10 at a time it might happen to 5-10% of them.
I think that my problem probably falls into this family of issues. I haven't seen it myself until v4.1.8 but now happens routinely and is about to drive me to find an alternative client.
Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.
Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.
I didn't - the NAS that I download to directly has over 18TB of free space. And it only started doing this after I started using v4.1.9.1 - it was OK in versions prior, though I do bypass some versions but I'm pretty sure it wasn't doing it under v4.1.7.
I would really like to get some developer attention on this because its a major usability issue.
Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.
Nope, over 3TB left on device
I'm having thoughts similar to those expressed by @cwarren-qc — lagging, flaky or faulty drives might be triggering this. OS is Windows 10 LTSC, qbt is v4.2.1.
I, too, notice these download being listed as 100% downloaded, and occasionally examining the individual files, you'll find that one file which is actually not 100% downloaded. Or, as is the case with the four "100% stalled" torrents that I'm re-re-checking at this moment, none of them list any file as less than 100% done, but the torrent information tab reads in plain English that less than the complete amount of pieces has been fetched.
So, in a few senses, they appear complete, while in at least one other, they do not. One has to wonder if there's any code in qbt making decisions based on those inaccurate reports and not looking at the accurate one?
I, too, am currently downloading a ton of torrents simultaneously, and for a few days now, also to three different drives, two of which were added via an external USB dual-HDD dock. Before this, I was downloading perhaps ten torrents at a time, to a SATA-based two-disk Windows-based RAID-0 array, and at that time I'd only experience this behaviour rarely. Now, with 75 simultaneously active torrents, it's constant.
One very interesting fact that seems clearly related to the issue is that qbt will repeatedly mark torrents that are being downloaded to the two USB drives as errorred, and checking the Event Viewer, I see lots of warnings about IO operations being retried on those two drives, but no IO errors are logged or otherwise reported, and eventually rechecking torrents suggest that no actual disk errors exist. It's possible that these disks might actually be flawed, but I can honestly say that prior to this I just did badblocks
runs on them with zero errors found, and then used them as intermediary storage for moving data around, with no errors, so it's doubtful. (Worthy of note is that the SATA-based RAID target drive doesn't get any torrents errorred by qbt).
What's more likely than actual disk errors, though, is that there's an issue with the logic board in the dock, or an issue with how Windows handles it, resulting in error situations occurring in the communications layer that are then recovered from, but somehow qbt gets triggered from these occurrences which exacerbates the issue that is described in this thread. (I've been using this dock for quite a while now, btw, with no issues, albeit on macOS and Linux and never on this machine, which FTR is also without USB3 ports, so the hub is in backwards compatibility mode on this machine, doing USB2 instead (hello, half-duplex 480 Mbps, welcome to the party)).
I don't know how deep the IO queue is for mass storage via USB (2.0) on Windows, or how it's handled, but if I had to come up with a non-synthetic scenario with which one could stress test it, I'd probably suggest downloading a bunch of torrents via gigabit internet, and to read and write to several drives on the same hub simultaneously. Perhaps make it use half-duplex USB 2.0 just for kicks. Which coincidentally is my exact setup.
So here's my thinking of what could be going on here. It's clearly not just my hardware setup, but knowing that these problems — or at least limitations forcing the storage subsystem into stress mode — exist here, I can at least postulate how events are unfolding.
I realise that was a super long-winded intro to get to the point, but I for one see a pattern in the reports here. Many simultaneous torrents and fast internet seem to trigger this issue, and with my own experiences in mind, I do suspect that there's some IO bottlenecking, queue issues, timeouts or error states occurring that cause qbt to (probably incorrectly) flag pieces as bad or missing, perhaps even disks as faulty. Either it then "forgets" to schedule those pieces for re-fetching, or it's actually thinking that there are disk errors so that it deliberately suspends IO, but is just not very vocal about what's going on.
If any devs are actually reading this thread (being a developer myself, I wouldn't fault them if they stopped reading after all the whining :D), I'd be happy to help out with diagnostics data, though be warned that this batch of torrents should be finished in not that many days.
The rest of you, do you also se "Errored" torrents in correlation with this issue? Also, when you see this behaviour again, could you check Event Viewer > Windows Logs > System, and check for recent disk warnings or similar storage-related warnings or errors?
Oh, and @Mike-EE — I hope you're not downloading directly to the target folders on that NAS of yours, considering the massive fragmentation that qbt/libtorrent cause on download drives (even with pre-allocation turned on).
@DanielSmedegaardBuus
I haven't had an issue with fragmentation on my NAS as the file system is good at self-optimization and avoiding fragmentation during writes. Regardless, I don't think that's the issue because this behavior is, for me, version dependent.
@arvidn this comment seems interesting: https://github.com/qbittorrent/qBittorrent/issues/7893#issuecomment-577556909
Could it be that there is a bug in flushing pieces to disk in libtorrent? Or perhaps it could prioritize flushing pieces for torrents that are near 100% (if it does not do that already)?
Alternatively, if a read/write operation is taking too long due to insufficient I/O subsystem performance, is it possible that it gets silently given up on? What are the alerts that qBIttorrent can catch to diagnose such an issue?
regardless of disk I/O bugs/dropped disk I/O ops. just because files or torrents are 100%, it doesn't necessarily mean they're done. 99.99% done is likely to round-up to 100%. In fact, any torrent with more than 200 pieces would have this property (one piece is < 0.5%). Perhaps this is unintuitive, and people would expect completion fractions to always be rounded down.
Regarding the issue; if this is indeed caused by disk errors, I would expect there would be some disk_error_alert
s being posted, with actual error codes. I haven't seen any reports of that though. It's also quite possible there's a bug in the bittorrent logic where some piece gets stuck in "no man's land". Where it wouldn't be eligible to be picked by the piece picker but still required to complete the download.
For example, perhaps if a piece fails the hash check it could get stuck under some circumstances. If a piece fails the hash check, there is some extra logic to ensure it's cleared from the disk thread(s) before being re-requested (to make sure none of the old blocks linger and cause it to fail again).
Does anyone see hash failures in the torrents that get stuck?
@arvidn
regardless of disk I/O bugs/dropped disk I/O ops. just because files or torrents are 100%, it doesn't necessarily mean they're done. 99.99% done is likely to round-up to 100%. In fact, any torrent with more than 200 pieces would have this property (one piece is < 0.5%). Perhaps this is unintuitive, and people would expect completion fractions to always be rounded down.
Is this something that comes from libtorrent? If so, it is definitely unintuitive. I would expect the completion fractions to always be rounded down, unless the rounding would yield 0%
(so as to prevent "download occurs but stuck at 0%
" confusion/reports). Put another way, no matter the size of the torrent, any progress at all should yield something different than 0% progress reported; anything other than all pieces completed should yield something different than 100% progress reported.
Regarding the issue; if this is indeed caused by disk errors, I would expect there would be some
disk_error_alerts
being posted, with actual error codes. I haven't seen any reports of that though.
I Ctrl+F'd disk_error_alert
here https://www.libtorrent.org/reference-Alerts.html, but did not find anything. Assuming you mean file_error_alert
, then qBittorrent does catch that, but I have not seen that mentioned in connection to these reports either, so I doubt it's the cause.
It's also quite possible there's a bug in the bittorrent logic where some piece gets stuck in "no man's land". Where it wouldn't be eligible to be picked by the piece picker but still required to complete the download.
For example, perhaps if a piece fails the hash check it could get stuck under some circumstances. If a piece fails the hash check, there is some extra logic to ensure it's cleared from the disk thread(s) before being re-requested (to make sure none of the old blocks linger and cause it to fail again).
Maybe, but then again,
Does anyone see hash failures in the torrents that get stuck?
I have not seen anything relating to that in these kinds of reports either.
Same here with 4.2.5. Stuck at 99.9%. Enable the recheck as mentioned above fix the issue.
MacOS Catalina. qBittorrent v4.2.5 - stalled status on 11.0%
@FranciscoPombal
Is this something that comes from libtorrent?
No, libtorrent tells you the number of pieces that are completed and the number of bytes completed. If you want to turn that into percent by dividing by the total number of pieces or total number of bytes, then you have to be careful with how you want to round the result. if you round to nearest, an almost-complete torrent can be 100%.
Assuming you mean file_error_alert, then qBittorrent does catch that, but I have not seen that mentioned in connection to these reports either, so I doubt it's the cause.
Some errors could be reported as torrent_error_alert as well.
@arvidn
Here's how qBittorrent calculates progress:
qreal TorrentHandleImpl::progress() const
{
if (!isChecking()) {
if (!m_nativeStatus.total_wanted)
return 0.;
if (m_nativeStatus.total_wanted_done == m_nativeStatus.total_wanted)
return 1.;
const qreal progress = static_cast<qreal>(m_nativeStatus.total_wanted_done) / m_nativeStatus.total_wanted;
Q_ASSERT((progress >= 0.f) && (progress <= 1.f));
return progress;
}
return m_nativeStatus.progress;
}
Where:
qreal
is just typedef double qreal;
m_nativeStatus
is lt::torrent_status
For the purposes of calculating the total torrent progress, the only notable manipulation that happens to this value is that it gets multiplied by 100.
, and then it is presumably rounded in some way or truncated when converting to string to only have at most 1 decimal place in the precentage.
I guess that even if there is nothing wrong with the final conversion to string, if the progress value is >= 0.999999995
, it gets rounded up to 1.0
, which will of course become 100
after multiplying by 100
and then show up as 100 %
.
I tested this with the following simple C program:
#include <stdio.h>
int main(int argc, char const *argv[])
{
double foo = .999999995;
double hundred = 100;
double bar = foo * hundred;
fprintf(stdout, "%f | %f\n", foo, bar);
return 0;
}
$ gcc --std=c11 test.c -o test
$ ./test
1.000000 | 100.000000 # note: using foo = .999999994 would yield 1.000000 | 99.999999 instead
In this screenshot (https://github.com/qbittorrent/qBittorrent/issues/7893#issuecomment-414024538), there are some torrents with size 1 < x < 2
GiB. A 2 GiB torrent with 10 bytes missing would presumably reproduce the problem: (2×1024³−10)÷(2×1024³) = 0.999999995
.
Thoughts?
No idea how to help, but i'm definitely getting items that are stalled, have plenty of seeders, but it's not talking to anyone. Some at 99.9% some at 0% some inbetween
it seems there are two issues:
torrents saying they are at 100% without being fully downloaded is confusing (presumably the inverse is also unintuitive, but perhaps less so. torrents that have bytes downloaded but are near 0% done are reported as 0%).
Torrents sometimes stall and stop download despite the user expecting them to download when there are seeds available.
@ericblade when you say "plenty of seeders", do you base than on anything other than what the qBT UI is telling you? Or is there a risk that it's a bug in reporting of numbers? If you see peers in the peer list that are seeds, that's a pretty safe indication of them being seeds.
I was looking at the main UI panel that was showing like anywhere from 2 to 12 or so seeders. I guess that low a number, I could be unable to communicate with any of them, though. Seems unlikely, but possible.
One of the 99.9% cases that I ran into was where it had downloaded the entirety (or very close to it) of the main file of a torrent, but failed to ever procure the 160 byte txt file that accompanied it. Marking that file as "do not download" then hitting "Force recheck" caused it to complete.
No idea on any of the others, though.
@arvidn commented on 2020. aug. 24. 21:06 CEST:
it seems there are two issues:
torrents saying they are at 100% without being fully downloaded is confusing (presumably the inverse is also unintuitive, but perhaps less so. torrents that have bytes downloaded but are near 0% done are reported as 0%).
Torrents sometimes stall and stop download despite the user expecting them to download when there are seeds available.
Mine is definitely the second. Which is invariably brought up by temporarily running out of space. The torrent just stalls without an error. After space is freed it it won't start. I pause/resume, but nothing happens. Also nothing happens if I restart the app, or recheck the torrent. I have to remove it, re-add it and wait for it to be checked, which is considerable pain.
This seems to be happening most frequently when disk i/o is busy on the hard drive the torrents are being written to either while being downloaded or when moving to another directory when completed. Using Windows 10. I have really not seen it happen at any other time on my computer.
I can pretty much guarantee this happening by having a couple of file copy/move operations happening on Windows while simultaneously having multiple torrents complete their downloads around the same time.
On my computer it doesn't happen frequently enough to really be anything but a rare annoyance.
what @mszo said. when i run out of disk space, everything halts with a "Stalled" .. and then the majority of it will never resume no matter what i do.
when libtorrent fails to write a piece to disk because the disk is full, the torrent is set to upload-only mode. i.e. it won't try to download anything, but it will keep uploading whatever it already has.
As long as the torrent is "auto managed" i.e. not force-started, it will clear the upload-only flag after 10 minutes. The idea is that if space has been freed up, it will resume. If it fails again, it will go back to upload-only mode and try again in another 10 minutes.
If the session is shut down in this state, the upload-only flag will probably be saved and restored when the session starts again, and there's a risk that it won't be cleared until 10 minutes after startup.
So, in the reports of it "never resumes", is that more than 10 minutes?
So, in the reports of it "never resumes", is that more than 10 minutes?
never appears to be "forever"
@arvidn commented on 2020. okt. 29. 11:32 CET:
when libtorrent fails to write a piece to disk because the disk is full, the torrent is set to upload-only mode. i.e. it won't try to download anything, but it will keep uploading whatever it already has.
As long as the torrent is "auto managed" i.e. not force-started, it will clear the upload-only flag after 10 minutes. The idea is that if space has been freed up, it will resume. If it fails again, it will go back to upload-only mode and try again in another 10 minutes.
How does force-start affect this? And why does it even do it?
If the session is shut down in this state, the upload-only flag will probably be saved and restored when the session starts again, and there's a risk that it won't be cleared until 10 minutes after startup.
So, in the reports of it "never resumes", is that more than 10 minutes?
I'm positive Ileft QB running for a good while in the background before deciding to take some action.
@arvidn Hold on. I forgot about the fact that this happens after the initial resume, which gets the torrent up to nearly completed, but one piece missing. Then it stalls. It wouldn't download the final piece, no matter what. I checked, the piece is in fact missing from the files. At least this is what happens here.
If the torrent is not finished, it should not be displayed as 100 percent. I have a torrent that's 98.9 percent complete under the Content tab but 100 percent complete on the main screen (torrent list). This needs to be addressed.
One 100% surefire way to trigger a download that'll get stuck at 99.9% is to start a torrent that won't fit on the target drive. It'll stall at whatever percentage it gets to once the disk is full (of course), and then after you clear enough space for it to complete, it will get stuck at 99.9% instead of completing. I have qbittorrent set up to automatically download from an RSS feed, and this happens all the time. Every time the disk fills up, each and every single torrent that at that point got stuck at that point will only download till 99.9% when I've freed up enough space for them to complete. A force re-check of their data will at that point bring them back to life, and they will continue to 100%.
Please provide the following information
qBittorrent version and Operating System
4.0.1 (64 bit) Windows 10 (10.0.1xxxx)
If on linux, libtorrent and Qt version
(type here)
What is the problem
When torrent download reaches 100%, it stalls instead of completing. Starting and stopping the torrent doesn't make any difference. It will only complete if I force it to recheck and rechecking takes awhile.
What is the expected behavior
It doesn't complete upon 100% like it used to. Problem only started when I upgraded to 4.0.1 from 3.x.x
Steps to reproduce
Run multiple torrents at the same time. When torrent reaches 100%, it doesn't complete. It just stays at stalled.
Extra info(if any)
Try to run with multiple torrents for higher chance of problems. 6 out of 7 torrents failed to complete for me. However if I run only 1 torrent, it "might" complete. (Some did, some didn't). I tried multiple batches of torrents to try and figure it out.