"ERROR_CANT_WAIT" is randomly produced

[GoogleCodeExporter]

What steps will reproduce the problem?

0. There is no method to reliably reproduce this; looks to be random, but only 
affects downloads.
1. Add a torrent to qBt, start downloading
2. If you're lucky the bug will manifest itself

What is the expected output? What do you see instead?
Torrent downloads normally.
Instead the following error is shown:
25/12/2012 22:31:58 - An I/O error occured, 'blanketyblank' paused.
25/12/2012 22:31:58 - Reason: blanketyblank file (C:/Users/Sarreq 
Teryx/Downloads/ut\blanketyblank.mkv.!qB) error: Used to indicate that an 
operation cannot continue without blocking for I/O

The error is always produced for the same file in torrent.

What version of the product are you using? On what operating system?

qBt git HEAD, Win7 x64
Was also reported for qBt 3.0.0 - 3.0.6
There's no info on whether this affects XP or not, assuming only NT >= 6 is 
affected; but this particular statement may be wrong.

Please provide any additional information below.

Rechecking and restarting torrent doesn't help.

The error is "ERROR_CANT_WAIT" and is defined by Windows.
My wild guess is that this may have something to do with OS Cache. After 
triggering the error, I've left my PC compiling heavy stuff overnight and 
couldn't reproduce that in the morning: all previously failing torrents were 
able to complete successfully. I suppose OS Cache for failing items has expired.

A similar workaround was posted on old qBt bugtracker:
1. Pause torrent
2. Zip file in question
3. Delete original
4. Unzip
5. Good to go

References:
Old qBt bugtracker: https://bugs.launchpad.net/qbittorrent/+bug/825092
Current qBt bugracker: https://github.com/qbittorrent/qBittorrent/issues/294

To be honest, I'm not even sure if this is an error. And if not can it be 
filtered out by file_error_alert?

P.S. We also have a report of "Overlapped I/O operation is in progress." 
(ERROR_IO_PENDING) on Windows Server 2008 R2 X64; I, personally, cannot confirm 
this atm, but there's a chance that ERROR_IO_PENDING and ERROR_CANT_WAIT are 
related.

Original issue reported on code.google.com by Daymansm...@gmail.com on 8 Jan 2013 at 1:22

[GoogleCodeExporter]

yeah, I would imagine these errors are related. thanks for the detailed report, 
I'll take a look

Original comment by arvid.no...@gmail.com on 9 Jan 2013 at 12:47

• Changed state: Accepted

[GoogleCodeExporter]

which version of libtorrent is this? I take it 0.16.x, right?

Original comment by arvid.no...@gmail.com on 9 Jan 2013 at 2:50

[GoogleCodeExporter]

Originally I bumped into this with 0.16.6, but also tried 0.16.5 and 0.16.4 
while I could still reproduce this.

Original comment by Daymansm...@gmail.com on 9 Jan 2013 at 8:48

[GoogleCodeExporter]

I would imagine it's related to files opened in OVERLAPPED mode, and 
specifically where the operation is launched with an overlapped structure, 
hence asking for it to be asynchronous.

My guess is that this is synonymous with ERROR_IO_PENDING so I've added 
ERROR_CANT_WAIT to be handled the same way.

You don't happen to know which actual system call causes this error, do you?

in libtorrent_aio I improved the disk I/O error reporting infrastructure to 
include the low-level operation that failed as well.

Original comment by arvid.no...@gmail.com on 10 Jan 2013 at 3:18

[GoogleCodeExporter]

> You don't happen to know which actual system call causes this error, do you?

Unfortunately I don't.

Building fails with the patch applied.
http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/branches/RC_0_16/src/fil
e.cpp?view=patch&r1=7840&r2=7839&pathrev=7840

src\file.cpp(1311) : error C2446: '==' : no conversion from 'long' to 'HANDLE'
        Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
src\file.cpp(1311) : error C2040: '==' : 'HANDLE' differs in levels of 
indirection from 'long'
src\file.cpp(1313) : error C2065: 'last_error' : undeclared identifier
src\file.cpp(1521) : error C2446: '==' : no conversion from 'long' to 'HANDLE'
        Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
src\file.cpp(1521) : error C2040: '==' : 'HANDLE' differs in levels of 
indirection from 'long'
src\file.cpp(1523) : error C2065: 'last_error' : undeclared identifier

Original comment by Daymansm...@gmail.com on 10 Jan 2013 at 12:55

[GoogleCodeExporter]

ah, yes. fixed.

Original comment by arvid.no...@gmail.com on 11 Jan 2013 at 4:50

[GoogleCodeExporter]

Does this still happen in 0.16.7 ?

Original comment by arvid.no...@gmail.com on 21 Jan 2013 at 5:28

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Still an issue with 0.16.7.
Finally got some traces.

---------------
0   libtorrent::file::writev    file.cpp    1593    0x7fed9e01d30   
1   libtorrent::default_storage::write_unaligned    storage.cpp 1345    0x7feda004542   
2   libtorrent::default_storage::readwritev storage.cpp 1219    0x7feda003bf8   
3   libtorrent::default_storage::writev storage.cpp 959 0x7feda002b0b   
4   libtorrent::piece_manager::write_impl   storage.cpp 1803    0x7feda006650   
5   libtorrent::disk_io_thread::flush_range disk_io_thread.cpp  607 0x7feda14ca2d   
6   libtorrent::disk_io_thread::flush_contiguous_blocks disk_io_thread.cpp  471 0x7
feda14c094  
7   libtorrent::disk_io_thread::thread_fun  disk_io_thread.cpp  2123    0x7feda152c95   
8   boost::_mfi::mf0<void,libtorrent::disk_io_thread>::operator()   mem_fn_template.
hpp 50  0x7feda1646fa   
9   boost::_bi::list1<boost::_bi::value<libtorrent::disk_io_thread * __ptr64> 
>::operator()<boost::_mfi::mf0<void,libtorrent::disk_io_thread>,boost::_bi::list
0>  bind.hpp    254 0x7feda164097   
10  boost::_bi::bind_t<void,boost::_mfi::mf0<void,libtorrent::disk_io_thread>,boo
st::_bi::list1<boost::_bi::value<libtorrent::disk_io_thread * __ptr64> > 
>::operator()   bind_template.hpp   21  0x7feda16286d   
11  boost::asio::detail::win_thread::func<boost::_bi::bind_t<void,boost::_mfi::mf
0<void,libtorrent::disk_io_thread>,boost::_bi::list1<boost::_bi::value<libtorren
t::disk_io_thread * __ptr64> > > >::run win_thread.hpp  117 0x7feda16268a   
12  boost::asio::detail::win_thread_function    win_thread.ipp  118 0x7fed9d92a4b   
13  beginthreadex   MSVCR100D       0x5fd472e5  
14  beginthreadex   MSVCR100D       0x5fd472a4  
15  BaseThreadInitThunk kernel32        0x765f652d  
16  RtlUserThreadStart  ntdll       0x76cec521  
---------------
And
---------------
0   libtorrent::file::writev    file.cpp    1593    0x7fed9e01d30   
1   libtorrent::default_storage::write_unaligned    storage.cpp 1345    0x7feda004542   
2   libtorrent::default_storage::readwritev storage.cpp 1219    0x7feda003bf8   
3   libtorrent::default_storage::writev storage.cpp 959 0x7feda002b0b   
4   libtorrent::piece_manager::write_impl   storage.cpp 1803    0x7feda006650   
5   libtorrent::disk_io_thread::flush_range disk_io_thread.cpp  607 0x7feda14ca2d   
6   libtorrent::disk_io_thread::thread_fun  disk_io_thread.cpp  2269    0x7feda153813   
7   boost::_mfi::mf0<void,libtorrent::disk_io_thread>::operator()   mem_fn_template.
hpp 50  0x7feda1646fa   
8   boost::_bi::list1<boost::_bi::value<libtorrent::disk_io_thread * __ptr64> 
>::operator()<boost::_mfi::mf0<void,libtorrent::disk_io_thread>,boost::_bi::list
0>  bind.hpp    254 0x7feda164097   
9   boost::_bi::bind_t<void,boost::_mfi::mf0<void,libtorrent::disk_io_thread>,boos
t::_bi::list1<boost::_bi::value<libtorrent::disk_io_thread * __ptr64> > 
>::operator()   bind_template.hpp   21  0x7feda16286d   
10  boost::asio::detail::win_thread::func<boost::_bi::bind_t<void,boost::_mfi::mf
0<void,libtorrent::disk_io_thread>,boost::_bi::list1<boost::_bi::value<libtorren
t::disk_io_thread * __ptr64> > > >::run win_thread.hpp  117 0x7feda16268a   
11  boost::asio::detail::win_thread_function    win_thread.ipp  118 0x7fed9d92a4b   
12  beginthreadex   MSVCR100D       0x5fd472e5  
13  beginthreadex   MSVCR100D       0x5fd472a4  
14  BaseThreadInitThunk kernel32        0x765f652d  
15  RtlUserThreadStart  ntdll       0x76cec521  

---------------

Original comment by Daymansm...@gmail.com on 22 Jan 2013 at 5:28

[GoogleCodeExporter]

I've tried ignoring those errors completely with attached patch. But it didn't 
go very well.
I was able to finish the torrent, but after reboot and recheck it had some 
pieces missing.

Roughly 1% of data was corrupt (~489 MiB) with 47.8 GiB torrent size and 16 MiB 
chunk size. This is around 30 corrupted chunks, scattered across 28 corrupted 
files with 43 files fully downloaded. Each file has a size of 533 MiB - 1.1 GiB.

Original comment by Daymansm...@gmail.com on 23 Jan 2013 at 9:00

Attachments:

• error.patch

[GoogleCodeExporter]

> I've tried ignoring those errors completely with attached patch. But it 
didn't go very well.
> I was able to finish the torrent, but after reboot and recheck it had some 
pieces missing.

> Roughly 1% of data was corrupt (~489 MiB) with 47.8 GiB torrent size and 16 
MiB chunk size. This is around 30 corrupted chunks, > scattered across 28 
corrupted files with 43 files fully downloaded. Each file has a size of 533 MiB 
- 1.1 GiB.

Strangely enough I cannot reproduce this with mingw64 build. Downloaded 40 GiB 
w/o issues.

I will remove my patch and revert original fix 
http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/branches/RC_0_16/src/fil
e.cpp?r1=7844&r2=7826&view=patch to see if mingw is affect at all.

This might get ugly if this is a compiler bug.

Original comment by Daymansm...@gmail.com on 23 Jan 2013 at 6:39

[GoogleCodeExporter]

my first attempt at handling this explicitly excluded mingw with #ifdefs, 
because that error code is not defined in the mingw headers and would fail to 
build for me otherwise.

Thanks for the traces, that's extremely helpful! Is that the only place you see 
this error happen?
It points to GetOverlappedResult() failing. I have a potential fix, could you 
try the attached patch?

Also, are you downloading onto an exotic filesystem by any chance? (by which I 
mean anything other than ntfs).

Original comment by arvid.no...@gmail.com on 24 Jan 2013 at 5:12

Attachments:

• wait.diff

[GoogleCodeExporter]

> Thanks for the traces, that's extremely helpful! Is that the only place you 
see this error happen?

Looks like it's the only place. I've set breakpoints at lines 1356 and 1593. 
But only the last one was triggered.

> I have a potential fix, could you try the attached patch?

Seems to be working. Running with original fix reverted and wait.diff applied. 
Downloaded around 50 GiB w/o any issues.

> Also, are you downloading onto an exotic filesystem by any chance? (by which 
I mean anything other than ntfs).

Nope. Just NTFS.
The only 'exotic' (in windows world) thing I can think of is that I'm running 
native 64-bit version. But the same happens with official qBittorrent builds, 
which are 32-bit.

Strange thing is that mingw was never affected (maybe because it's using posix 
threads?).

Original comment by Daymansm...@gmail.com on 24 Jan 2013 at 8:55

[GoogleCodeExporter]

It appears to be caused by the bug mentioned in one of the comments at the 
bottom of:

   http://msdn.microsoft.com/en-us/library/windows/desktop/ms683209(v=vs.85).aspx

Here's what that report says:

Windows 7: Resolves a race condition where a multi-threaded app using 
GetOverlappedResult can return without resetting the event in the overlapped 
structure, causing the next call to this function to return prematurely. 

Windows Vista (default): Provides the behavior with the race condition that 
applications may have a dependency on. Applications wishing to avoid this race 
prior to the Windows 7 behavior should wait on the overlapped event and when 
signaled, call GetOverlappedResult with bWait == FALSE. "

Original comment by arvid.no...@gmail.com on 25 Jan 2013 at 5:48

[GoogleCodeExporter]

I've checked in that patch now. thanks for the report.

Original comment by arvid.no...@gmail.com on 25 Jan 2013 at 5:52

• Changed state: Fixed

[GoogleCodeExporter]

Unfortunately this bug hit me again. On the same spot with the same backtrace. 
I'm attaching the backtrace with the contents of OVERLAPPED structure if this 
is of any help.

I've also tried to embed compatibility manifest in both torrent.dll and 
qbittorrent.exe as described in 
http://msdn.microsoft.com/en-us/library/dd371711%28VS.85%29.aspx w/o any 
success.

The command I use to build libtorrent is:
bjam -j4 -q --toolset=msvc --prefix=%INST_DIR% boost=system boost-link=shared 
link=shared runtime-link=shared variant=release debug-symbols=off 
resolve-countries=on full-stats=on export-extra=off ipv6=on dht-support=on 
character-set=unicode geoip=static encryption=openssl windows-version=vista 
threading=multi address-model=64 host-os=windows target-os=windows 
embed-manifest=on architecture=x86 warnings=off warnings-as-errors=off 
inlining=full optimization=speed "cflags=/O2 /GL /favor:blend" 
"linkflags=/NOLOGO /OPT:REF /OPT:ICF=5 /LTCG" 
"include=%BUILDROOT%\OpenSSL\OpenSSL64\include" 
"include=%BUILDROOT%\Boost\Boost64\include" 
"library-path=%BUILDROOT%\OpenSSL\OpenSSL64\lib" 
"library-path=%BUILDROOT%\Boost\Boost64\lib" "define=BOOST_ALL_NO_LIB" install

----

qBittorrent is built with BOOST_ASIO_ENABLE_CANCELIO, which is not enabled by 
default, but is enabled in libtorrent Jamfile so I turn it on.

I use windows-version=vista with libtorrent and define _WIN32_WINNT=0x0600 when 
building qBittorrent, so they match.

Original comment by Daymansm...@gmail.com on 8 Feb 2013 at 9:21

Attachments:

• lt_trace.txt

[GoogleCodeExporter]

are you using RC_0_16 or 0.16.7?

RC_0_16 has one more fix which I'm pretty confident in.

Original comment by arvid.no...@gmail.com on 9 Feb 2013 at 9:52

[GoogleCodeExporter]

I'm using 0.16.7 patched with 
http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/branches/RC_0_16/src/fil
e.cpp?r1=7844&r2=7932

Are there other fixes beyond file.cpp?

Original comment by Daymansm...@gmail.com on 9 Feb 2013 at 11:02

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I've tried RC_0_16 with same results.

I've also tried checking WaitForSingleObject return value, and it's always ok - 
zero.

Original comment by Daymansm...@gmail.com on 9 Feb 2013 at 12:41

[GoogleCodeExporter]

that was the patch I had in mind. I think I may have made two check-ins, 
because I missed a few uses of GetOverlappedResult() the first time.

It's odd that GetOverlappedResult() would return can't wait when it's not asked 
to wait (i.e. last argument is false).

Original comment by arvid.no...@gmail.com on 9 Feb 2013 at 10:05

[GoogleCodeExporter]

0.16.8 still has the issue.

So far only the workaround from old qBt bugtracker helps: archive the file in 
question, delete original, unpack, recheck and go.

Doesn't help:

1. Restart
2. Disabling antivirus (well, I had to try)
3. Using 'disable_os_cache'

I'll try to use preallocation instead of sparse files, maybe this will help, 
I'm just out of ideas.

Original comment by Daymansm...@gmail.com on 10 Feb 2013 at 9:29

[GoogleCodeExporter]

Huh. Using preallocation instead of sparse files actually throws 
ERROR_IO_PENDING, so those two cases are really connected.

I made a trace (along with a trace from another thread, which may or may be not 
helpful), but I cannot pinpoint the exact source of file alert produced, 
probably comes from a thread. I'd be glad to set breakpoints, but I need the 
exact location.

My guess would be that two or more threads are trying to write same blocks.

Original comment by Daymansm...@gmail.com on 13 Feb 2013 at 2:27

Attachments:

• trace.txt

[GoogleCodeExporter]

Original comment by arvid.no...@gmail.com on 13 Feb 2013 at 5:37

• Changed state: Accepted

[GoogleCodeExporter]

I realized I forgot to handle errors properly in my latest patch. Here's a fix 
for that which I'm putting in right now:

Index: src/file.cpp
===================================================================
--- src/file.cpp    (revision 8011)
+++ src/file.cpp    (working copy)
@@ -1342,7 +1342,12 @@
                CloseHandle(ol.hEvent);
                return -1;
            }
-           WaitForSingleObject(ol.hEvent, INFINITE);
+           if (WaitForSingleObject(ol.hEvent, INFINITE) == WAIT_FAILED)
+           {
+               ec.assign(GetLastError(), get_system_category());
+               CloseHandle(ol.hEvent);
+               return -1;
+           }
            DWORD num_read;
            if (GetOverlappedResult(m_file_handle, &ol, &num_read, false) == 0)
            {
@@ -1582,7 +1587,12 @@
                CloseHandle(ol.hEvent);
                return -1;
            }
-           WaitForSingleObject(ol.hEvent, INFINITE);
+           if (WaitForSingleObject(ol.hEvent, INFINITE) == WAIT_FAILED)
+           {
+               ec.assign(GetLastError(), get_system_category());
+               CloseHandle(ol.hEvent);
+               return -1;
+           }
            DWORD num_written;
            if (GetOverlappedResult(m_file_handle, &ol, &num_written, false) == 0)
            {

Also, setting a breakpoint in storage::set_error() may pin-point where the 
error is coming from.

Original comment by arvid.no...@gmail.com on 13 Feb 2013 at 5:59

[GoogleCodeExporter]

I'm attaching the trace with BP in storage.cpp before the link gets lost in IRC 
logs.

Original comment by Daymansm...@gmail.com on 13 Feb 2013 at 7:35

Attachments:

• trace.txt

[GoogleCodeExporter]

I'm attaching stats from disk-stats=on logging=default build

Original comment by Daymansm...@gmail.com on 6 Aug 2013 at 10:49

Attachments:

• qBittorrent64.7z

[GoogleCodeExporter]

We are still getting users reporting this problem with 0.16.17: 
http://forum.deluge-torrent.org/viewtopic.php?f=12&t=48473&p=201695#p201695

What do you need in order to fix this issue?

Original comment by caluml...@gmail.com on 23 Sep 2014 at 7:32

[GoogleCodeExporter]

The solution works for me, but please fix this annoying issue.

Original comment by john.a...@gmail.com on 24 Sep 2014 at 8:20

[GoogleCodeExporter]

which solution works for you?

my understanding is that all proposed solutions in this thread made it into 
libtorrent quite a while ago.

Original comment by arvid.no...@gmail.com on 24 Sep 2014 at 8:27

[GoogleCodeExporter]

I think he meant that the archive/unpack solution (workaround).

As reported above, Deluge is still experiencing this issue in latest versions 
using 0.16.17.

Original comment by jackofal...@gmail.com on 17 Oct 2014 at 6:19

[GoogleCodeExporter]

Client: 1.3.9

libtorrent: 0.16.18.0

Still happening... 

Original comment by kokau...@gmail.com on 11 Nov 2014 at 2:30

[GoogleCodeExporter]

Client: Deluge 1.3.10
libtorrent: 0.16.18
Still happening.

Original comment by old...@gmail.com on 17 Nov 2014 at 12:34

[GoogleCodeExporter]

people who experience this, do you have disk_io_write_mode set to 
disable_os_cache by any chance? or disable_os_cache_for_aligned_files?

If so, would you mind trying out setting it to  enable_os_cache and see if that 
solves it?

Original comment by arvid.no...@gmail.com on 18 Nov 2014 at 3:12

[GoogleCodeExporter]

How do you change these options in windows?

Original comment by old...@gmail.com on 18 Nov 2014 at 7:32

[GoogleCodeExporter]

That is the case in Deluge: 
http://git.deluge-torrent.org/deluge/tree/deluge/core/core.py?h=1.3-stable#n102

To change it you would need to download and modify that file then drop it into 
relevant location of Deluge install.

Arvid what is the recommendation for that setting in >=0.16? I know it was 
changed to prevent memory issue in 0.15: 
https://code.google.com/p/libtorrent/issues/detail?id=166 

Original comment by caluml...@gmail.com on 18 Nov 2014 at 9:24

[GoogleCodeExporter]

Actually just remembered that using the ltconfig plugin should allow you to 
change these settings too. 

Original comment by caluml...@gmail.com on 18 Nov 2014 at 9:28

[GoogleCodeExporter]

the main reason for these settings to exist was because of windows causing 
excessive caching of files. This was fixed in 0.16.x.

http://blog.libtorrent.org/2012/05/windows-disk-cache/

Original comment by arvid.no...@gmail.com on 18 Nov 2014 at 9:36

[GoogleCodeExporter]

I am trying to use Ltconfig since I don't know how to use git link.
Using ltconfig I have line disk_io_write_mode with value 2. Should I try value 
1 instead?
Thanks

Original comment by old...@gmail.com on 18 Nov 2014 at 9:39

[GoogleCodeExporter]

No it should be 0

http://www.rasterbar.com/products/libtorrent/manual.html#session-customization

                enable_os_cache = 0,
                disable_os_cache_for_aligned_files = 1,
                disable_os_cache = 2

Original comment by caluml...@gmail.com on 18 Nov 2014 at 9:52

[GoogleCodeExporter]

Offtopic: And because of the ever-growing RAM usage because of OS 
caching(Windows) qbittorrent gave in and exposes an option for users to disable 
os cache. The problem(growing RAM usage) mainly manifested with seeders that 
had a lot of traffic.

Original comment by hammered...@gmail.com on 18 Nov 2014 at 12:22

[GoogleCodeExporter]

Setting enable_os_cache through ltconfig apparently solved the problem. I hope 
that now I won't have a RAM eater as well.

Original comment by old...@gmail.com on 18 Nov 2014 at 2:16

[GoogleCodeExporter]

if you're on a recent 0.16.x version, you shouldn't have that problem. (the 
details are in the blog post, and it was fixed in the 0.16.x branch). When this 
was fixed, the default value for both of those were changed to 0 as well.

The option (and feature) has been removed from trunk, but still in 1.0.x.

I wonder if it would make sense to rip it out of 0.16.x and up too.

Original comment by arvid.no...@gmail.com on 18 Nov 2014 at 10:09

[GoogleCodeExporter]

No arvid. Users of qbt with 0.16.16(or 0.16.17) reported that ever-growing RAM 
problem.
Usually Windows 7 users that also had very active seeding traffic. Builds with 
disable_os_cache seemed to solve the problem, so I exposed it as a checkbox in 
the advanced settings.

Original comment by hammered...@gmail.com on 18 Nov 2014 at 10:34

[GoogleCodeExporter]

hammered999: do you have a link to a recent report?

Original comment by arvid.no...@gmail.com on 19 Nov 2014 at 5:54

[GoogleCodeExporter]

@arvidn here are some really recent ones:
Comment 1: 
https://github.com/qbittorrent/qBittorrent/issues/1457#issuecomment-68320007
Comment 2: 
https://github.com/qbittorrent/qBittorrent/issues/1457#issuecomment-68439062
qbt v3.1.11 uses 0.16.18+svn10417

Original comment by hammered...@gmail.com on 31 Dec 2014 at 1:22