0.16.9 hangs on torrent completion

[GoogleCodeExporter]

I am experiencing this on Windows XP sp2 32bit with qBittorrent 3.0.9(and git 
master), libtorrent 0.16.9.

What happens? When a torrent finishes downloading:
1. If there are torrents downloading or are in the queue they cant make new 
connections to peers. They are stalling. From observing the alerts posted new 
connection attempts time out.
2. If the user wants to exit the application, it hangs. I have hunted down the 
hang and it is caused by the libttorrent::session's desctructor blockage. If I 
exit the application before a torrent finishes then it exits cleanly.
3. This happens with libtorrent 0.16.9, RC_0_16 and trunk. If I use 0.16.8 it 
works. Also I observe that many udp_error_alerts are post with this message: 
"UDP error: No connection could be made because the target machine actively 
refused it from: x.x.x.x". 0.16.8 still has this alerts but are fewer.
4. When a stalled torrent occurs(see number 1) I observe in the alerts that all 
of a sudden libtorrent discovers many peers from "local peer discovery". The 
funny thing is that I am alone in the home network.
5. The save_resume_data on the finished torrent doesn't post the 
save_resume_data_alert even after waiting for 30 seconds. It works if I do the 
same before the torrent finishes.

Does this indicate a deadlocked libtorrent worker thread?

Here is the call stack of the first 3 threads of the program after a torrent 
has finished:

Thread 0:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   SetWindowTextW  USER32      0x77d1bbfe  
2   QEventDispatcherWin32::processEvents    qbittorrent     0x9ac297    
3   QGuiEventDispatcherWin32::processEvents qbittorrent     0x62064b    
4   QEventLoop::processEvents   qbittorrent     0x9a104a    
5   QEventLoop::exec    qbittorrent     0x9a112d    
6   QCoreApplication::exec  qbittorrent     0x93d05b    
7   QApplication::exec  qbittorrent     0x4f55f9    
8   main    main.cpp    369 0x4a260b    
9   WinMain qbittorrent     0x9eb164    
10  __tmainCRTStartup   crt0.c  263 0xd64bbb    
11  RegisterWaitForInputIdle    kernel32        0x7c816d4f  

Thread 1:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   libtorrent::overlapped_t::wait  qbittorrent     0xac9d70    
3   libtorrent::file::close qbittorrent     0xacbbec    
4   libtorrent::file::~file qbittorrent     0xacc435    
5   std::_Tree<std::_Tmap_traits<std::pair<void 
*,int>,libtorrent::file_pool::lru_file_entry,std::less<std::pair<void *,int> 
>,std::allocator<std::pair<std::pair<void *,int> const 
,libtorrent::file_pool::lru_file_entry> >,0> >::erase   qbittorrent     0xb88c5a    
6   libtorrent::file_pool::release  qbittorrent     0xb89100    
7   libtorrent::default_storage::release_files  qbittorrent     0xa72209    
8   libtorrent::disk_io_thread::thread_fun  qbittorrent     0xb72e38    
9   boost::asio::detail::win_thread_function    qbittorrent     0xaaaf58    
10  _callthreadstartex  threadex.c  348 0xd662d3    
11  _threadstartex  threadex.c  326 0xd6637b    
12  GetModuleFileNameA  kernel32        0x7c80b50b  

Thread 2:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   boost::asio::detail::win_iocp_io_service::timer_thread_function::operator() qb
ittorrent       0xab14e4    
3   boost::asio::detail::win_thread_function    qbittorrent     0xaaaf58    
4   _callthreadstartex  threadex.c  348 0xd662d3    
5   _threadstartex  threadex.c  326 0xd6637b    
6   GetModuleFileNameA  kernel32        0x7c80b50b  

This is the callstack of the fist 3 threads after I try to exit qbt (which 
hangs on libtorrent::session's destructor)

Thread 0:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   Sleep   kernel32        0x7c802451  
2   libtorrent::sleep   qbittorrent     0xb2e15b    
3   libtorrent::alert_manager::wait_for_alert   qbittorrent     0xc0c008    
4   libtorrent::aux::session_impl::wait_for_alert   qbittorrent     0xb2f065    
5   libtorrent::session::wait_for_alert qbittorrent     0xa377e2    
6   QBtSession::saveFastResumeData  qbtsession.cpp  1668    0x414208    
7   QBtSession::~QBtSession qbtsession.cpp  186 0x40d83a    
8   QBtSession::`scalar deleting destructor'    qbittorrent     0x40d74c    
9   QBtSession::drop    qbtsession.cpp  2849    0x41acbf    
10  MainWindow::~MainWindow mainwindow.cpp  394 0x4b6a89    
11  main    main.cpp    369 0x4a2622    
12  WinMain qbittorrent     0x9eb164    
13  __tmainCRTStartup   crt0.c  263 0xd64bbb    
14  RegisterWaitForInputIdle    kernel32        0x7c816d4f  

Thread 1:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   libtorrent::overlapped_t::wait  qbittorrent     0xac9d70    
3   libtorrent::file::close qbittorrent     0xacbbec    
4   libtorrent::file::~file qbittorrent     0xacc435    
5   std::_Tree<std::_Tmap_traits<std::pair<void 
*,int>,libtorrent::file_pool::lru_file_entry,std::less<std::pair<void *,int> 
>,std::allocator<std::pair<std::pair<void *,int> const 
,libtorrent::file_pool::lru_file_entry> >,0> >::erase   qbittorrent     0xb88c5a    
6   libtorrent::file_pool::release  qbittorrent     0xb89100    
7   libtorrent::default_storage::release_files  qbittorrent     0xa72209    
8   libtorrent::disk_io_thread::thread_fun  qbittorrent     0xb72e38    
9   boost::asio::detail::win_thread_function    qbittorrent     0xaaaf58    
10  _callthreadstartex  threadex.c  348 0xd662d3    
11  _threadstartex  threadex.c  326 0xd6637b    
12  GetModuleFileNameA  kernel32        0x7c80b50b  

Thread 2:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   boost::asio::detail::win_iocp_io_service::timer_thread_function::operator() qb
ittorrent       0xab14e4    
3   boost::asio::detail::win_thread_function    qbittorrent     0xaaaf58    
4   _callthreadstartex  threadex.c  348 0xd662d3    
5   _threadstartex  threadex.c  326 0xd6637b    
6   GetModuleFileNameA  kernel32        0x7c80b50b  

Using boost 1.53.0, openssl 1.0.1, zlib 1.2.7. Everything is linked 
statically(even the msvc runtime). Compiled with msvc2008

Original issue reported on code.google.com by hammered...@gmail.com on 7 Apr 2013 at 5:19

[GoogleCodeExporter]

Also if I add a new torrent after a torrent has finished, it is stuck in the 
"checking" status.

Original comment by hammered...@gmail.com on 7 Apr 2013 at 5:33

[GoogleCodeExporter]

Sorry the hang on libtorrent::~session() doesn't show up in the previous 
callstacks since the wait_for_alert() hadn't timed out yet. Here is the 
callstack of the first 4 threads of qbt after the wait_for_alert() has timed 
out and qbt tries to shutdown libtorrent in order to exit.

Thread 0:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForMultipleObjects  kernel32        0x7c809c86  
2   boost::asio::detail::win_thread::join   qbittorrent     0xaafe25    
3   libtorrent::disk_io_thread::join    qbittorrent     0xb6bdd2    
4   libtorrent::aux::session_impl::~session_impl    qbittorrent     0xb613fc    
5   boost::detail::sp_counted_impl_p<libtorrent::aux::session_impl>::dispose    qbitt
orrent      0xa3b95f    
6   libtorrent::session::~session   qbittorrent     0xa4f37f    
7   libtorrent::session::`scalar deleting destructor'   qbittorrent     0x40da23    
8   QBtSession::~QBtSession qbtsession.cpp  205 0x40d8ec    
9   QBtSession::`scalar deleting destructor'    qbittorrent     0x40d74c    
10  QBtSession::drop    qbtsession.cpp  2851    0x41accb    
11  MainWindow::~MainWindow mainwindow.cpp  394 0x4b698c    
12  main    main.cpp    369 0x4a2636    
13  WinMain qbittorrent     0x9eaf85    
14  __tmainCRTStartup   crt0.c  263 0xd64a3b    
15  RegisterWaitForInputIdle    kernel32        0x7c816d4f  

Thread 1:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   libtorrent::overlapped_t::wait  qbittorrent     0xac9b80    
3   libtorrent::file::close qbittorrent     0xacb9fc    
4   libtorrent::file::~file qbittorrent     0xacc245    
5   std::_Tree<std::_Tmap_traits<std::pair<void 
*,int>,libtorrent::file_pool::lru_file_entry,std::less<std::pair<void *,int> 
>,std::allocator<std::pair<std::pair<void *,int> const 
,libtorrent::file_pool::lru_file_entry> >,0> >::erase   qbittorrent     0xb88a7a    
6   libtorrent::file_pool::release  qbittorrent     0xb88f20    
7   libtorrent::default_storage::release_files  qbittorrent     0xa72039    
8   libtorrent::disk_io_thread::thread_fun  qbittorrent     0xb72c58    
9   boost::asio::detail::win_thread_function    qbittorrent     0xaaad88    
10  _callthreadstartex  threadex.c  348 0xd66153    
11  _threadstartex  threadex.c  326 0xd661fb    
12  GetModuleFileNameA  kernel32        0x7c80b50b  

Thread 2:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   WaitForSingleObject kernel32        0x7c802542  
2   boost::asio::detail::win_iocp_io_service::timer_thread_function::operator() qb
ittorrent       0xab1304    
3   boost::asio::detail::win_thread_function    qbittorrent     0xaaad88    
4   _callthreadstartex  threadex.c  348 0xd66153    
5   _threadstartex  threadex.c  326 0xd661fb    
6   GetModuleFileNameA  kernel32        0x7c80b50b  

Thread 3:
0   KiFastSystemCallRet ntdll       0x7c91eb94  
1   boost::asio::detail::win_iocp_io_service::do_one    qbittorrent     0xabbe59    
2   boost::asio::detail::win_iocp_io_service::run   qbittorrent     0xabc9b8    
3   boost::asio::detail::win_thread_function    qbittorrent     0xaaad88    
4   GetModuleFileNameA  kernel32        0x7c80b50b  
5               0x12f834    
6               0x12f824    
7               0x1d82400   
8               0x7ffdb000  
9               0xffffffff8a236600  
10              0x260ffc0   
11              0xffffffff89592cd0  
12              0xffffffffffffffff  
13  FindAtomW   kernel32        0x7c8399f3  
14  GetModuleFileNameA  kernel32        0x7c80b518  

Original comment by hammered...@gmail.com on 9 Apr 2013 at 12:09

[GoogleCodeExporter]

it looks like the disk thread is stuck in closing files. Is it possible that 
the filesystem you're writing to may block in close() to complete allocation of 
the file on disk? Mac OS does that since it doesn't support sparse files. Are 
you using sparse files on windows? (iirc, it's on by default).

Original comment by arvid.no...@gmail.com on 9 Apr 2013 at 4:36

[GoogleCodeExporter]

Filesystem is NTFS.

>block in close() to complete allocation of the file on disk?

I am not using preallocation, so he files are sparsed. This behavior happens 
when a torrent has finished downloading(I will test with multifile torrents 
later). The close() doesn't hang if I exit qbt in the middle of the download.

And I repeat here, since it may have slipped you, this does not happen in 
0.16.8.

Is there anything else I can do to provide more info or to test?

Original comment by hammered...@gmail.com on 9 Apr 2013 at 8:23

[GoogleCodeExporter]

I see. Auditing this code a bit, I think this patch may handle some error cases 
a bit better. Could you please try it?

Index: src/file.cpp
===================================================================
--- src/file.cpp    (revision 8302)
+++ src/file.cpp    (working copy)
@@ -858,7 +858,8 @@
        }
        int wait(HANDLE file, error_code& ec)
        {
-           if (WaitForSingleObject(ol.hEvent, INFINITE) == WAIT_FAILED)
+           if (handle != INVALID_HANDLE_VALUE
+               && WaitForSingleObject(ol.hEvent, INFINITE) == WAIT_FAILED)
            {
                ec.assign(GetLastError(), get_system_category());
                return -1;
@@ -1000,10 +1001,10 @@
            DWORD temp;
            bool use_overlapped = m_open_mode & no_buffer;
            overlapped_t ol;
-           ::DeviceIoControl(m_file_handle, FSCTL_SET_SPARSE, 0, 0
+           BOOL ret = ::DeviceIoControl(m_file_handle, FSCTL_SET_SPARSE, 0, 0
                , 0, 0, &temp, use_overlapped ? &ol.ol : NULL);
            error_code error;
-           if (use_overlapped)
+           if (use_overlapped && ret == FALSE && GetLastError() == ERROR_IO_PENDING)
                ol.wait(m_file_handle, error);
        }
 #else // TORRENT_WINDOWS
@@ -1228,7 +1229,7 @@
        BOOL ret = DeviceIoControl(file, FSCTL_QUERY_ALLOCATED_RANGES, (void*)&in, sizeof(in)
            , out, sizeof(out), &returned_bytes, overlapped ? &ol.ol : NULL);

-       if (overlapped)
+       if (overlapped && ret == FALSE && GetLastError() == ERROR_IO_PENDING)
        {
            error_code ec;
            returned_bytes = ol.wait(file, ec);
@@ -1274,10 +1275,10 @@
            DWORD temp;
            FILE_SET_SPARSE_BUFFER b;
            b.SetSparse = FALSE;
-           int ret = ::DeviceIoControl(m_file_handle, FSCTL_SET_SPARSE, &b, sizeof(b)
+           BOOL ret = ::DeviceIoControl(m_file_handle, FSCTL_SET_SPARSE, &b, sizeof(b)
                , 0, 0, &temp, use_overlapped ? &ol.ol : NULL);
            error_code ec;
-           if (use_overlapped)
+           if (use_overlapped && ret == FALSE && GetLastError() == ERROR_IO_PENDING)
            {
                ol.wait(m_file_handle, ec);
            }

Original comment by arvid.no...@gmail.com on 10 Apr 2013 at 3:39

[GoogleCodeExporter]

>error: 'handle' : undeclared identifier

I assume that you meant "ol.hEvent" and I am currently testing with that.

Original comment by hammered...@gmail.com on 10 Apr 2013 at 12:05

[GoogleCodeExporter]

It works.
It also works if for 'handle' is use 'file' instead!?!?!

Original comment by hammered...@gmail.com on 10 Apr 2013 at 1:27

[GoogleCodeExporter]

oh, right. I mean ol.hEvent.

Thanks for testing! I'll put this patch in and it will be included in the 
0.16.10 release.

Original comment by arvid.no...@gmail.com on 10 Apr 2013 at 3:01

[GoogleCodeExporter]

Original comment by arvid.no...@gmail.com on 10 Apr 2013 at 11:38

• Changed state: Fixed