qbittorrent / qBittorrent

qBittorrent BitTorrent client
https://www.qbittorrent.org
Other
28.34k stars 3.99k forks source link

Unable to complete torrent. Stays at "Stalled" status when 100% #7893

Open anonymous82 opened 6 years ago

anonymous82 commented 6 years ago

Please provide the following information

qBittorrent version and Operating System

4.0.1 (64 bit) Windows 10 (10.0.1xxxx)

If on linux, libtorrent and Qt version

(type here)

What is the problem

When torrent download reaches 100%, it stalls instead of completing. Starting and stopping the torrent doesn't make any difference. It will only complete if I force it to recheck and rechecking takes awhile.

What is the expected behavior

It doesn't complete upon 100% like it used to. Problem only started when I upgraded to 4.0.1 from 3.x.x

Steps to reproduce

Run multiple torrents at the same time. When torrent reaches 100%, it doesn't complete. It just stays at stalled.

Extra info(if any)

Try to run with multiple torrents for higher chance of problems. 6 out of 7 torrents failed to complete for me. However if I run only 1 torrent, it "might" complete. (Some did, some didn't). I tried multiple batches of torrents to try and figure it out.

slrslr commented 6 years ago

qbittorrent / Tools / Options / Advanced there is option "Recheck on completion", maybe it can work around that? Btw, it looks similar to this: https://github.com/qbittorrent/qBittorrent/issues/7850

skarlsen commented 6 years ago

I also encountered this issue today when downloading 6 torrents at a time. The first one completed like normal, and after starting the three next ones, I started two more that had a higher priority to me so I paused the 3 torrents started in the "second batch". The two torrents I started most recently finished like normal, while the 3 that I had paused all suffered from this issue when they "completed". I resumed downloading the 3 at the same time.

When looking at the status bar in the main torrent list, labeled "Done" in the header, the three affected torrents showed 100%. However, when inspecting the individual files in the "content" tab, the biggest file in all three torrents showed as 99.9% completed. Going back to the "general" tab, I observed that they were missing 5, 7 and 7 pieces out of roughly 4500 pieces (the total number of pieces was similar for all 3 torrents).

Upon forcing a recheck, all 3 torrents completed instantly.

I currently have 600 torrents in my list, with many more completed in the past and on other devices, and I have never encountered this issue before updating to 4.0.1.

redfellow commented 6 years ago

Happends to me as well. Edit: Transmission with the Remote GUI addon has mostly the same feel and features as Qbittorrent, and doesn't have these ludicrous bugs that the devs are incabable of fixing even when they have been known for over a year.

Suggest you guys give it a go if you are getting as annoyed, as I did.

japrine commented 6 years ago

Started happening to me as well (just randomly) on 4.0.3. If I pause-resume-force resume etc it doesn't help. I have even done a recheck on one and it didn't finish, I had to exit and reopen qbit to get it finished. Also odd was the "run once complete" command I have didn't run when I reopened to make it finish. On each one overall status is 100% but one of the files will be 99.9% (with tons of seeders).

Edit: ok force recheck is working to restart them now. And they will "run once complete". Before it may not have worked because I was already downloading more then my que allowed.

anonymous82 commented 6 years ago

Well after updating to latest version, I'm happy to say it doesn't really happen anymore.

Before this, forcing a recheck was the only way to get it complete as I mentioned previously.

beeeeswax commented 6 years ago

This is still happening in 4.1.1.

pressRtowin commented 6 years ago

Still happening on latest (4.1.1) x64 running on windows server 2012. Very annoying. Will point out that suggestions above to use the "recheck on completion" option won't work as the torrent is technically not complete. I'm mostly dealing with larger torrents (50-300GB) and every time I've checked one of these stalled torrents, there's anywhere from 1 - 50MB that hasn't downloaded. Rechecking these manually is very tedious, not to mention the unnecessary I/O (and TIME) it's been eating up.

pressRtowin commented 6 years ago

Upgraded to 4.1.2 today and it doesn't seem to be happening anymore... But it could just be dumb luck. I'll keep y'all updated.

jonathanmcdougall commented 6 years ago

Happens on 4.1.2, been happening forever. Restarting qbittorrent completes them.

qb

hiragashi commented 6 years ago

bugs for this date back 5+ years they wont be fixing it anytime soon. the only true fix is ditch qbt and use something that works properly

redfellow commented 6 years ago

https://transmissionbt.com/ + https://sourceforge.net/projects/transgui/ is an alternative with a familiar feel to it.

wwwarezwally commented 6 years ago

Yep I get this exact problem as well and more annoyingly I now have the RSS re-downloading the same torrents (only certain ones) OVER AND OVER AND OVER AND OVER and they just keep coming back as unread even though they were downloaded/deleted a week ago

bathrobehero commented 5 years ago

Still an issue. The more I use qbt, the more half a decade old issues I find but the alternatives are even worse. Force recheck solves is, I'm guessing it's a caching issue of some sort.

tugboat-maguire commented 5 years ago

Still having this issue as well

mzso commented 5 years ago

In recent times I only got a stall at 99% when I ran out of space. After I resolve it the torrent is guaranteed to get stuck at 99%

sergiotapia commented 5 years ago

Still happening just now. I'm going to use something else unfortunately.

For all would be Googler's just select all the affect torrents, and Force Recheck.

vectorrilke commented 5 years ago

Happens to me as well in 4.1.5. Stalled on 100%, disk usage is also 100%. Seems to be happening randomly, or I did not find any pattern.

AbsenceJam commented 5 years ago

4.1.6 on Windows, still occuring.

I have "Recheck on completion" enabled.

beeeeswax commented 5 years ago

I love how this has been reported and re-reported for years without so much as an acknowledgement from the developers. First class, no, really.

As in last comment, this still happens in 4.1.6.

Shout out to all 5.x users with this issue from the future. Is that twat Trump still in charge?

sergiotapia commented 5 years ago

Maintaining open source software is shit work, and soul-croushing after the honeymoon period. If you don't like it submit a patch or move to a different software. Don't insult the creators.

imasuridondai commented 5 years ago

In 4.1.7 it happens much more frequently that it was in 4.1.5 (4.1.6 was too buggy to use it)

pietervw commented 4 years ago

Still happening to me on 4.20, even with "Recheck on completion" turned on. Quitting and re-opening the client doesn't move them past stalled (100%).

cwarren-qc commented 4 years ago

Happen for me with 4.2.1. But, in my case, I think I know what was the source of the problem. I had a lot of downloads at the same time and my hard drive was not able to keep up with downloading/validating pieces (and other tasks I was running on the same drive). The torrents were stalled at 100% done, but there were green parts in the "General \ Progress" bar (i.e. partial pieces). I "paused" all the others torrents and qbittorrent was able to validate the 'green pieces' and complete the download. There might be a bug where a '100% downloaded but partially validated' torrent is too low in the priority list.

tsubus commented 4 years ago

happened to me as well. it usually doesn't, but yesterday i downloaded a lot of big torrents at the same time (which is a very rare occurrence) and a few torrents got stalled at 100% overnight. a manual force recheck fixed them.

japrine commented 4 years ago

tsbus - Same for me, Its been a ongoing issue for me for a long time on multiple computers/versions but its rare except when downloading allot at once. If I download 100 torrents 1 at a time it might not happen at all, if I download 100 - 10 at a time it might happen to 5-10% of them.

Mike-EE commented 4 years ago

11677

I think that my problem probably falls into this family of issues. I haven't seen it myself until v4.1.8 but now happens routinely and is about to drive me to find an alternative client.

mzso commented 4 years ago

Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.

Mike-EE commented 4 years ago

Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.

I didn't - the NAS that I download to directly has over 18TB of free space. And it only started doing this after I started using v4.1.9.1 - it was OK in versions prior, though I do bypass some versions but I'm pretty sure it wasn't doing it under v4.1.7.

I would really like to get some developer attention on this because its a major usability issue.

tsubus commented 4 years ago

Neither of you ran out of disk space? It looks like there might be a number of similar issues. To me the torrents fail to complete if I run out of space in the meanwhile. After I free up space the stop before completion. But not at 100% but typically 99% (or less for smaller torrents), one piece short.

Nope, over 3TB left on device

DanielSmedegaardBuus commented 4 years ago

I'm having thoughts similar to those expressed by @cwarren-qc — lagging, flaky or faulty drives might be triggering this. OS is Windows 10 LTSC, qbt is v4.2.1.

I, too, notice these download being listed as 100% downloaded, and occasionally examining the individual files, you'll find that one file which is actually not 100% downloaded. Or, as is the case with the four "100% stalled" torrents that I'm re-re-checking at this moment, none of them list any file as less than 100% done, but the torrent information tab reads in plain English that less than the complete amount of pieces has been fetched.

So, in a few senses, they appear complete, while in at least one other, they do not. One has to wonder if there's any code in qbt making decisions based on those inaccurate reports and not looking at the accurate one?

I, too, am currently downloading a ton of torrents simultaneously, and for a few days now, also to three different drives, two of which were added via an external USB dual-HDD dock. Before this, I was downloading perhaps ten torrents at a time, to a SATA-based two-disk Windows-based RAID-0 array, and at that time I'd only experience this behaviour rarely. Now, with 75 simultaneously active torrents, it's constant.

One very interesting fact that seems clearly related to the issue is that qbt will repeatedly mark torrents that are being downloaded to the two USB drives as errorred, and checking the Event Viewer, I see lots of warnings about IO operations being retried on those two drives, but no IO errors are logged or otherwise reported, and eventually rechecking torrents suggest that no actual disk errors exist. It's possible that these disks might actually be flawed, but I can honestly say that prior to this I just did badblocks runs on them with zero errors found, and then used them as intermediary storage for moving data around, with no errors, so it's doubtful. (Worthy of note is that the SATA-based RAID target drive doesn't get any torrents errorred by qbt).

What's more likely than actual disk errors, though, is that there's an issue with the logic board in the dock, or an issue with how Windows handles it, resulting in error situations occurring in the communications layer that are then recovered from, but somehow qbt gets triggered from these occurrences which exacerbates the issue that is described in this thread. (I've been using this dock for quite a while now, btw, with no issues, albeit on macOS and Linux and never on this machine, which FTR is also without USB3 ports, so the hub is in backwards compatibility mode on this machine, doing USB2 instead (hello, half-duplex 480 Mbps, welcome to the party)).

I don't know how deep the IO queue is for mass storage via USB (2.0) on Windows, or how it's handled, but if I had to come up with a non-synthetic scenario with which one could stress test it, I'd probably suggest downloading a bunch of torrents via gigabit internet, and to read and write to several drives on the same hub simultaneously. Perhaps make it use half-duplex USB 2.0 just for kicks. Which coincidentally is my exact setup.

So here's my thinking of what could be going on here. It's clearly not just my hardware setup, but knowing that these problems — or at least limitations forcing the storage subsystem into stress mode — exist here, I can at least postulate how events are unfolding.

I realise that was a super long-winded intro to get to the point, but I for one see a pattern in the reports here. Many simultaneous torrents and fast internet seem to trigger this issue, and with my own experiences in mind, I do suspect that there's some IO bottlenecking, queue issues, timeouts or error states occurring that cause qbt to (probably incorrectly) flag pieces as bad or missing, perhaps even disks as faulty. Either it then "forgets" to schedule those pieces for re-fetching, or it's actually thinking that there are disk errors so that it deliberately suspends IO, but is just not very vocal about what's going on.

If any devs are actually reading this thread (being a developer myself, I wouldn't fault them if they stopped reading after all the whining :D), I'd be happy to help out with diagnostics data, though be warned that this batch of torrents should be finished in not that many days.

The rest of you, do you also se "Errored" torrents in correlation with this issue? Also, when you see this behaviour again, could you check Event Viewer > Windows Logs > System, and check for recent disk warnings or similar storage-related warnings or errors?

Oh, and @Mike-EE — I hope you're not downloading directly to the target folders on that NAS of yours, considering the massive fragmentation that qbt/libtorrent cause on download drives (even with pre-allocation turned on).

Mike-EE commented 4 years ago

@DanielSmedegaardBuus

I haven't had an issue with fragmentation on my NAS as the file system is good at self-optimization and avoiding fragmentation during writes. Regardless, I don't think that's the issue because this behavior is, for me, version dependent.

FranciscoPombal commented 4 years ago

@arvidn this comment seems interesting: https://github.com/qbittorrent/qBittorrent/issues/7893#issuecomment-577556909

Could it be that there is a bug in flushing pieces to disk in libtorrent? Or perhaps it could prioritize flushing pieces for torrents that are near 100% (if it does not do that already)?

Alternatively, if a read/write operation is taking too long due to insufficient I/O subsystem performance, is it possible that it gets silently given up on? What are the alerts that qBIttorrent can catch to diagnose such an issue?

arvidn commented 4 years ago

regardless of disk I/O bugs/dropped disk I/O ops. just because files or torrents are 100%, it doesn't necessarily mean they're done. 99.99% done is likely to round-up to 100%. In fact, any torrent with more than 200 pieces would have this property (one piece is < 0.5%). Perhaps this is unintuitive, and people would expect completion fractions to always be rounded down.

Regarding the issue; if this is indeed caused by disk errors, I would expect there would be some disk_error_alerts being posted, with actual error codes. I haven't seen any reports of that though. It's also quite possible there's a bug in the bittorrent logic where some piece gets stuck in "no man's land". Where it wouldn't be eligible to be picked by the piece picker but still required to complete the download.

For example, perhaps if a piece fails the hash check it could get stuck under some circumstances. If a piece fails the hash check, there is some extra logic to ensure it's cleared from the disk thread(s) before being re-requested (to make sure none of the old blocks linger and cause it to fail again).

Does anyone see hash failures in the torrents that get stuck?

FranciscoPombal commented 4 years ago

@arvidn

regardless of disk I/O bugs/dropped disk I/O ops. just because files or torrents are 100%, it doesn't necessarily mean they're done. 99.99% done is likely to round-up to 100%. In fact, any torrent with more than 200 pieces would have this property (one piece is < 0.5%). Perhaps this is unintuitive, and people would expect completion fractions to always be rounded down.

Is this something that comes from libtorrent? If so, it is definitely unintuitive. I would expect the completion fractions to always be rounded down, unless the rounding would yield 0% (so as to prevent "download occurs but stuck at 0%" confusion/reports). Put another way, no matter the size of the torrent, any progress at all should yield something different than 0% progress reported; anything other than all pieces completed should yield something different than 100% progress reported.

Regarding the issue; if this is indeed caused by disk errors, I would expect there would be some disk_error_alerts being posted, with actual error codes. I haven't seen any reports of that though.

I Ctrl+F'd disk_error_alert here https://www.libtorrent.org/reference-Alerts.html, but did not find anything. Assuming you mean file_error_alert, then qBittorrent does catch that, but I have not seen that mentioned in connection to these reports either, so I doubt it's the cause.

It's also quite possible there's a bug in the bittorrent logic where some piece gets stuck in "no man's land". Where it wouldn't be eligible to be picked by the piece picker but still required to complete the download.

For example, perhaps if a piece fails the hash check it could get stuck under some circumstances. If a piece fails the hash check, there is some extra logic to ensure it's cleared from the disk thread(s) before being re-requested (to make sure none of the old blocks linger and cause it to fail again).

Maybe, but then again,

Does anyone see hash failures in the torrents that get stuck?

I have not seen anything relating to that in these kinds of reports either.

artik commented 4 years ago

Same here with 4.2.5. Stuck at 99.9%. Enable the recheck as mentioned above fix the issue.

gr8den commented 4 years ago

MacOS Catalina. qBittorrent v4.2.5 - stalled status on 11.0%

arvidn commented 4 years ago

@FranciscoPombal

Is this something that comes from libtorrent?

No, libtorrent tells you the number of pieces that are completed and the number of bytes completed. If you want to turn that into percent by dividing by the total number of pieces or total number of bytes, then you have to be careful with how you want to round the result. if you round to nearest, an almost-complete torrent can be 100%.

Assuming you mean file_error_alert, then qBittorrent does catch that, but I have not seen that mentioned in connection to these reports either, so I doubt it's the cause.

Some errors could be reported as torrent_error_alert as well.

FranciscoPombal commented 4 years ago

@arvidn

Here's how qBittorrent calculates progress:

qreal TorrentHandleImpl::progress() const
{
    if (!isChecking()) {
        if (!m_nativeStatus.total_wanted)
            return 0.;

        if (m_nativeStatus.total_wanted_done == m_nativeStatus.total_wanted)
            return 1.;

        const qreal progress = static_cast<qreal>(m_nativeStatus.total_wanted_done) / m_nativeStatus.total_wanted;
        Q_ASSERT((progress >= 0.f) && (progress <= 1.f));
        return progress;
    }

    return m_nativeStatus.progress;
}

Where:

For the purposes of calculating the total torrent progress, the only notable manipulation that happens to this value is that it gets multiplied by 100., and then it is presumably rounded in some way or truncated when converting to string to only have at most 1 decimal place in the precentage.

I guess that even if there is nothing wrong with the final conversion to string, if the progress value is >= 0.999999995, it gets rounded up to 1.0, which will of course become 100 after multiplying by 100 and then show up as 100 %.

I tested this with the following simple C program:

#include <stdio.h>

int main(int argc, char const *argv[])
{
    double foo = .999999995;
    double hundred = 100;
    double bar = foo * hundred;

    fprintf(stdout, "%f | %f\n", foo, bar);

    return 0;
}
$ gcc --std=c11 test.c -o test
$ ./test
1.000000 | 100.000000 # note: using foo = .999999994 would yield 1.000000 | 99.999999 instead

In this screenshot (https://github.com/qbittorrent/qBittorrent/issues/7893#issuecomment-414024538), there are some torrents with size 1 < x < 2 GiB. A 2 GiB torrent with 10 bytes missing would presumably reproduce the problem: (2×1024³−10)÷(2×1024³) = 0.999999995.

Thoughts?

ericblade commented 4 years ago

No idea how to help, but i'm definitely getting items that are stalled, have plenty of seeders, but it's not talking to anyone. Some at 99.9% some at 0% some inbetween

arvidn commented 4 years ago

it seems there are two issues:

  1. torrents saying they are at 100% without being fully downloaded is confusing (presumably the inverse is also unintuitive, but perhaps less so. torrents that have bytes downloaded but are near 0% done are reported as 0%).

  2. Torrents sometimes stall and stop download despite the user expecting them to download when there are seeds available.

@ericblade when you say "plenty of seeders", do you base than on anything other than what the qBT UI is telling you? Or is there a risk that it's a bug in reporting of numbers? If you see peers in the peer list that are seeds, that's a pretty safe indication of them being seeds.

ericblade commented 4 years ago

I was looking at the main UI panel that was showing like anywhere from 2 to 12 or so seeders. I guess that low a number, I could be unable to communicate with any of them, though. Seems unlikely, but possible.
One of the 99.9% cases that I ran into was where it had downloaded the entirety (or very close to it) of the main file of a torrent, but failed to ever procure the 160 byte txt file that accompanied it. Marking that file as "do not download" then hitting "Force recheck" caused it to complete. No idea on any of the others, though.

mzso commented 4 years ago

@arvidn commented on 2020. aug. 24. 21:06 CEST:

it seems there are two issues:

  1. torrents saying they are at 100% without being fully downloaded is confusing (presumably the inverse is also unintuitive, but perhaps less so. torrents that have bytes downloaded but are near 0% done are reported as 0%).

  2. Torrents sometimes stall and stop download despite the user expecting them to download when there are seeds available.

Mine is definitely the second. Which is invariably brought up by temporarily running out of space. The torrent just stalls without an error. After space is freed it it won't start. I pause/resume, but nothing happens. Also nothing happens if I restart the app, or recheck the torrent. I have to remove it, re-add it and wait for it to be checked, which is considerable pain.

gorbachev commented 4 years ago

This seems to be happening most frequently when disk i/o is busy on the hard drive the torrents are being written to either while being downloaded or when moving to another directory when completed. Using Windows 10. I have really not seen it happen at any other time on my computer.

I can pretty much guarantee this happening by having a couple of file copy/move operations happening on Windows while simultaneously having multiple torrents complete their downloads around the same time.

On my computer it doesn't happen frequently enough to really be anything but a rare annoyance.

ericblade commented 4 years ago

what @mszo said. when i run out of disk space, everything halts with a "Stalled" .. and then the majority of it will never resume no matter what i do.

arvidn commented 4 years ago

when libtorrent fails to write a piece to disk because the disk is full, the torrent is set to upload-only mode. i.e. it won't try to download anything, but it will keep uploading whatever it already has.

As long as the torrent is "auto managed" i.e. not force-started, it will clear the upload-only flag after 10 minutes. The idea is that if space has been freed up, it will resume. If it fails again, it will go back to upload-only mode and try again in another 10 minutes.

If the session is shut down in this state, the upload-only flag will probably be saved and restored when the session starts again, and there's a risk that it won't be cleared until 10 minutes after startup.

So, in the reports of it "never resumes", is that more than 10 minutes?

ericblade commented 4 years ago

So, in the reports of it "never resumes", is that more than 10 minutes?

never appears to be "forever"

mzso commented 4 years ago

@arvidn commented on 2020. okt. 29. 11:32 CET:

when libtorrent fails to write a piece to disk because the disk is full, the torrent is set to upload-only mode. i.e. it won't try to download anything, but it will keep uploading whatever it already has.

As long as the torrent is "auto managed" i.e. not force-started, it will clear the upload-only flag after 10 minutes. The idea is that if space has been freed up, it will resume. If it fails again, it will go back to upload-only mode and try again in another 10 minutes.

How does force-start affect this? And why does it even do it?

If the session is shut down in this state, the upload-only flag will probably be saved and restored when the session starts again, and there's a risk that it won't be cleared until 10 minutes after startup.

So, in the reports of it "never resumes", is that more than 10 minutes?

I'm positive Ileft QB running for a good while in the background before deciding to take some action.

mzso commented 4 years ago

@arvidn Hold on. I forgot about the fact that this happens after the initial resume, which gets the torrent up to nearly completed, but one piece missing. Then it stalls. It wouldn't download the final piece, no matter what. I checked, the piece is in fact missing from the files. At least this is what happens here.

Knocks commented 3 years ago

If the torrent is not finished, it should not be displayed as 100 percent. I have a torrent that's 98.9 percent complete under the Content tab but 100 percent complete on the main screen (torrent list). This needs to be addressed.

DanielSmedegaardBuus commented 3 years ago

One 100% surefire way to trigger a download that'll get stuck at 99.9% is to start a torrent that won't fit on the target drive. It'll stall at whatever percentage it gets to once the disk is full (of course), and then after you clear enough space for it to complete, it will get stuck at 99.9% instead of completing. I have qbittorrent set up to automatically download from an RSS feed, and this happens all the time. Every time the disk fills up, each and every single torrent that at that point got stuck at that point will only download till 99.9% when I've freed up enough space for them to complete. A force re-check of their data will at that point bring them back to life, and they will continue to 100%.