msvc 2010 can sometimes produce vector iterator not dereferenceable error in init()

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Construct an MSVC 2010 project using simple_client.cpp as a base
2.Above the main function in simple_client.cpp add the following code from 
dump_torrent.cpp:

#include boost/filesystem/operations.hpp

int Dump_Torrent(int argc, char* argv[])
{
    using namespace libtorrent;
    using namespace boost::filesystem;

    if (argc != 2)
    {
        std::cerr << "usage: dump_torrent torrent-file" << std::endl;
        return 1;
    }
#if BOOST_VERSION < 103400
    boost::filesystem::path::default_name_check(boost::filesystem::no_check);
#endif

#ifndef BOOST_NO_EXCEPTIONS
    try
    {
#endif
        int size = (int)file_size(path(argv[1]));
        if (size > 10 * MB_SIZE)
        {
            std::cerr << "file too big (" << size << "), aborting" << std::endl;
            return 1;
        }

        std::vector<char> buf(size);
        std::ifstream(argv[1], std::ios_base::binary).read(&buf[0], size);
        entry e = bdecode(&buf[0], &buf[0] + buf.size());

        torrent_info t(e);

#ifndef BOOST_NO_EXCEPTIONS
    }
    catch (std::exception& e)
    {
        std::cout << e.what() << std::endl;
    }
#endif

    return 0;

}

3. After constructing torrent_info t, add the following code to construct an 
iterator to output the web_seeds

        for (std::vector<web_seed_entry>::const_iterator j = t.web_seeds().begin(); j != t.web_seeds().end(); ++j)
        {
            std::cout << "\t" << j->url << std::endl;
        }
4. Create a trackerless torrent with a single url_seed
5. Compile and run the program with the path to the torrent file as the only 
argument and pre-processor definitions _DEBUG, _CONSOLE, and TORRENT_DEBUG in 
addition to the standard setup

What is the expected output? What do you see instead?
I expect, since we are not calling Dump_Torrent that the program would silently 
download the files from the url_seed until return is pressed.
What I am seeing is an error stating that a "vector iterator is not 
dereferenceable".  Tracing through the stack I notice that the error is fired 
from add_torrent's init() function and not in my code, however if you comment 
out the code added in step 3 the program will run flawlessly.

What version of the product are you using? On what operating system?
libtorrent v0.16.9, boost v1.53.0, openssl v1.0.1e, msvc 2010 on Win7 x64 
(program compiled as x86)

Please provide any additional information below.
I'd like to think this is something wrong with my end but seeing as how the 
program errors out with the inclusion of code that isn't even called I can't 
rule out compiler error.

Original issue reported on code.google.com by r...@thepit.org on 18 Apr 2013 at 11:13

Attachments:

• simple_client.cpp

[GoogleCodeExporter]

The .cpp files referenced in steps 1 and 2 are from the examples directory in 
libtorrent's source.  Step 2's code snippet shows the code as it should look 
after renaming the main function from dump_torrent.cpp to Dump_Torrent and 
removing all but the most basic instructions to allow the code in step 3 to 
work.

Original comment by r...@thepit.org on 18 Apr 2013 at 11:20

[GoogleCodeExporter]

Are you also building libtorrent with TORRENT_DEBUG defined?

could you post the stack trace to this assert?

It sounds like an ABI incompatibility.

Original comment by arvid.no...@gmail.com on 19 Apr 2013 at 7:53

[GoogleCodeExporter]

I have removed TORRENT_DEBUG from my preprocessor definitions and get linker 
errors so I assume that libtorrent was built with this defined. I used the 
default build options.

The stack trace is as follows:
>   msvcr100d.dll!_crt_debugger_hook(int _Reserved)  Line 62    C
    msvcr100d.dll!_call_reportfault(int nDbgHookCode, unsigned long dwExceptionCode, unsigned long dwExceptionFlags)  Line 167 + 0x9 bytes  C++
    msvcr100d.dll!_invoke_watson(const wchar_t * pszExpression, const wchar_t * pszFunction, const wchar_t * pszFile, unsigned int nLine, unsigned int pReserved)  Line 155 + 0xe bytes C++
    msvcr100d.dll!_invalid_parameter(const wchar_t * pszExpression, const wchar_t * pszFunction, const wchar_t * pszFile, unsigned int nLine, unsigned int pReserved)  Line 111 C++
    Sample_Client.exe!std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > >::operator*()  Line 71 + 0x50 bytes C++
    Sample_Client.exe!std::list<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> >::_Insert<std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > >(std::_List_const_iterator<std::_List_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _Where, std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _First, std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _Last, std::forward_iterator_tag __formal)  Line 1042 + 0x8 bytes    C++
    Sample_Client.exe!std::list<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> >::insert<std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > >(std::_List_const_iterator<std::_List_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _Where, std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _First, std::_Vector_const_iterator<std::_Vector_val<libtorrent::web_seed_entry,std::allocator<libtorrent::web_seed_entry> > > _Last)  Line 1005 C++
    Sample_Client.exe!libtorrent::torrent::init()  Line 1622 + 0xbd bytes   C++
    Sample_Client.exe!libtorrent::torrent::start()  Line 893    C++
    Sample_Client.exe!libtorrent::aux::session_impl::add_torrent(const libtorrent::add_torrent_params & p, boost::system::error_code & ec)  Line 4963   C++
    Sample_Client.exe!boost::_mfi::mf2<libtorrent::torrent_handle,libtorrent::aux::session_impl,libtorrent::add_torrent_params const &,boost::system::error_code &>::operator()(libtorrent::aux::session_impl * p, const libtorrent::add_torrent_params & a1, boost::system::error_code & a2)  Line 280 + 0x1c bytes    C++
    Sample_Client.exe!boost::_bi::list3<boost::_bi::value<libtorrent::aux::session_impl *>,boost::_bi::value<libtorrent::add_torrent_params>,boost::reference_wrapper<boost::system::error_code> >::operator()<libtorrent::torrent_handle,boost::_mfi::mf2<libtorrent::torrent_handle,libtorrent::aux::session_impl,libtorrent::add_torrent_params const &,boost::system::error_code &>,boost::_bi::list0>(boost::_bi::type<libtorrent::torrent_handle> __formal, boost::_mfi::mf2<libtorrent::torrent_handle,libtorrent::aux::session_impl,libtorrent::add_torrent_params const &,boost::system::error_code &> & f, boost::_bi::list0 & a, boost::_bi::type<libtorrent::torrent_handle> __formal)  Line 382 + 0x4a bytes   C++
    Sample_Client.exe!boost::_bi::bind_t<libtorrent::torrent_handle,boost::_mfi::mf2<libtorrent::torrent_handle,libtorrent::aux::session_impl,libtorrent::add_torrent_params const &,boost::system::error_code &>,boost::_bi::list3<boost::_bi::value<libtorrent::aux::session_impl *>,boost::_bi::value<libtorrent::add_torrent_params>,boost::reference_wrapper<boost::system::error_code> > >::operator()()  Line 20 + 0x23 bytes    C++
    Sample_Client.exe!boost::detail::function::function_obj_invoker0<boost::_bi::bind_t<libtorrent::torrent_handle,boost::_mfi::mf2<libtorrent::torrent_handle,libtorrent::aux::session_impl,libtorrent::add_torrent_params const &,boost::system::error_code &>,boost::_bi::list3<boost::_bi::value<libtorrent::aux::session_impl *>,boost::_bi::value<libtorrent::add_torrent_params>,boost::reference_wrapper<boost::system::error_code> > >,libtorrent::torrent_handle>::invoke(boost::detail::function::function_buffer & function_obj_ptr)  Line 132 + 0xc bytes  C++
    Sample_Client.exe!boost::function0<libtorrent::torrent_handle>::operator()()  Line 767 + 0x18 bytes C++
    Sample_Client.exe!libtorrent::fun_ret<libtorrent::torrent_handle>(libtorrent::torrent_handle * ret, bool * done, libtorrent::condition * e, boost::asio::detail::win_mutex * m, boost::function<libtorrent::torrent_handle __cdecl(void)> f)  Line 303 + 0xc bytes  C++
    Sample_Client.exe!boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > >::operator()<void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list0>(boost::_bi::type<void> __formal, void (libtorrent::torrent_handle *, bool *, libtorrent::condition *, boost::asio::detail::win_mutex *, boost::function<libtorrent::torrent_handle __cdecl(void)>)* & f, boost::_bi::list0 & a, boost::_bi::type<void> __formal)  Line 525 + 0x8b bytes C++
    Sample_Client.exe!boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > >::operator()()  Line 21   C++
    Sample_Client.exe!boost::asio::asio_handler_invoke<boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > >(boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > function, ...)  Line 65  C++
    Sample_Client.exe!boost_asio_handler_invoke_helpers::invoke<boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > >,boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > >(boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > & function, boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > & context)  Line 39 + 0x26 bytes  C++
    Sample_Client.exe!boost::asio::detail::completion_handler<boost::_bi::bind_t<void,void (__cdecl*)(libtorrent::torrent_handle *,bool *,libtorrent::condition *,boost::asio::detail::win_mutex *,boost::function<libtorrent::torrent_handle __cdecl(void)>),boost::_bi::list5<boost::_bi::value<libtorrent::torrent_handle *>,boost::_bi::value<bool *>,boost::_bi::value<libtorrent::condition *>,boost::_bi::value<boost::asio::detail::win_mutex *>,boost::_bi::value<boost::function<libtorrent::torrent_handle __cdecl(void)> > > > >::do_complete(boost::asio::detail::win_iocp_io_service * owner, boost::asio::detail::win_iocp_operation * base, const boost::system::error_code & __formal, const boost::system::error_code & __formal)  Line 67 + 0xd bytes    C++
    Sample_Client.exe!boost::asio::detail::win_iocp_operation::complete(boost::asio::detail::win_iocp_io_service & owner, const boost::system::error_code & ec, unsigned int bytes_transferred)  Line 45 + 0x18 bytes   C++
    Sample_Client.exe!boost::asio::detail::win_iocp_io_service::do_one(bool block, boost::system::error_code & ec)  Line 405    C++
    Sample_Client.exe!boost::asio::detail::win_iocp_io_service::run(boost::system::error_code & ec)  Line 161 + 0xe bytes   C++
    Sample_Client.exe!boost::asio::io_service::run(boost::system::error_code & ec)  Line 67 C++
    Sample_Client.exe!libtorrent::aux::session_impl::main_thread()  Line 4713   C++
    Sample_Client.exe!boost::_mfi::mf0<void,libtorrent::aux::session_impl>::operator()(libtorrent::aux::session_impl * p)  Line 50  C++
    Sample_Client.exe!boost::_bi::list1<boost::_bi::value<libtorrent::aux::session_impl *> >::operator()<boost::_mfi::mf0<void,libtorrent::aux::session_impl>,boost::_bi::list0>(boost::_bi::type<void> __formal, boost::_mfi::mf0<void,libtorrent::aux::session_impl> & f, boost::_bi::list0 & a, boost::_bi::type<void> __formal)  Line 254   C++
    Sample_Client.exe!boost::_bi::bind_t<void,boost::_mfi::mf0<void,libtorrent::aux::session_impl>,boost::_bi::list1<boost::_bi::value<libtorrent::aux::session_impl *> > >::operator()()  Line 21  C++
    Sample_Client.exe!boost::asio::detail::win_thread::func<boost::_bi::bind_t<void,boost::_mfi::mf0<void,libtorrent::aux::session_impl>,boost::_bi::list1<boost::_bi::value<libtorrent::aux::session_impl *> > > >::run()  Line 117    C++
    Sample_Client.exe!boost::asio::detail::win_thread_function(void * arg)  Line 118    C++
    msvcr100d.dll!_callthreadstartex()  Line 314 + 0xf bytes    C
    msvcr100d.dll!_threadstartex(void * ptd)  Line 297  C
    kernel32.dll!76fc33aa()     
    [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]  
    ntdll.dll!77d79ef2()    
    ntdll.dll!77d79ec5()

Original comment by r...@thepit.org on 19 Apr 2013 at 3:55

[GoogleCodeExporter]

Ok I think I found part of the problem.  VS 2010 defaulted my character set to 
Multi-Byte instead of Unicode.  After changing that I no longer get the vector 
errors within the library by simply referencing the web_seeds vector.

I still have an issue with comparing the iterator I am incrementing in that for 
loop with the web_seeds().end() value.  It appears to never equal that value 
before it runs off the end and causes dereferencing errors.  I can work around 
that by setting the loop condition to j < web_seeds().end(), but that implies 
that the increment operator is miscalculating the size of a web_seed and any 
more than 1 entry will cause unknown results.  Could this be an architecture 
issue?  Perhaps the lib is building to x64 while I'm trying to build the exe as 
x86.

Original comment by r...@thepit.org on 23 Apr 2013 at 6:49

[GoogleCodeExporter]

It does sound like some ABI incompatibility issue. You may want to make sure 
libtorrent and your project are both built with compatible (ideally identical) 
compiler switches.

the layout of web_seed_entry does not depend on any libtorrent configuration 
option. If the two translation units have different understanding of its size 
and layout, I can only think if compiler switches as being the reason.

It's surprising that the unicode setting would make any difference as well. 
You're referring to the UNICODE define, right? I can't think of a reason that 
would alter the layout of web_seed_entry either.

Original comment by arvid.no...@gmail.com on 24 Apr 2013 at 4:36

[GoogleCodeExporter]

Could it be an ABI incompatibility of std::vector<> and its iterators? msvc may 
use debug iterators when _DEBUG is defined and if libtorrent was built without 
_DEBUG, that could cause an issue like this.

Original comment by ar...@rasterbar.com on 27 Apr 2013 at 7:38

[GoogleCodeExporter]

So as far as the debug build is concerned it looks like the issue is an ABI 
incompatibility with std::vector<> and its iterators, but not because "_DEBUG" 
was defined, it's because "DEBUG" wasn't defined.  I went though the command 
line that VS2010 was generating and noticed this wasn't included in the 
preprocessor defines on the compiler.  As soon as I defined it the debug builds 
began working as expected.

Now I'm just having issues with the release builds in the same area, only now 
the release iterators don't catch the overrun and try to operate over garbage.  
I've tried to force identical switches between boost, libtorrent, and my 
project (leaving out openssl support for now), but still no joy.  It may be 
that I'm not building the boost binaries to the same specs that libtorrent 
interprets them when it builds off of boost source, but I'm not sure how that 
affects std::vector<>.

Compiler switches: /I"C:\libtorrent\libtorrent-rasterbar-0.16.9\include" 
/I"C:\Boost\boost_1_53_0" /Zi /nologo /W3 /WX- /O2 /Oi /GL /D "WIN32" /D 
"NDEBUG" /D "BOOST_ASIO_SEPARATE_COMPILATION" /D "CONSOLE" /D "_CONSOLE" /D 
"_UNICODE" /D "UNICODE" /Gm- /EHsc /MT /GS /fp:precise /Zc:wchar_t /Zc:forScope 
/Fp"x64\Release_64\MiniBT.pch" /Fa"x64\Release_64\" /Fo"x64\Release_64\" 
/Fd"x64\Release_64\vc100.pdb" /Gd /errorReport:queue 

Linker Switches: /NOLOGO /LIBPATH:"C:\Boost\boost_1_53_0\stage64r\lib" 
/LIBPATH:"C:\libtorrent\libtorrent-rasterbar-0.16.9\bin\msvc-10.0\release\addres
s-model-64\architecture-x86\boost-source\link-static\runtime-link-static\threadi
ng-multi\windows-version-win7" "libtorrent.lib" "kernel32.lib" "user32.lib" 
"gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" 
"ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /MANIFEST 
/ManifestFile:"x64\Release_64\MiniBT.exe.intermediate.manifest" /ALLOWISOLATION 
/MANIFESTUAC:"level='asInvoker' uiAccess='false'" /OPT:REF /OPT:ICF /LTCG 
/TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X64 /ERRORREPORT:QUEUE 

Original comment by r...@thepit.org on 30 Apr 2013 at 4:03

[GoogleCodeExporter]

boost build is very good at solving this problem, but enforcing link-compatible 
compiler switches across all dependencies. my assumption would be that the 
DEBUG macro controls whether iterators have asserts in them or not.

Original comment by arvid.no...@gmail.com on 28 Oct 2013 at 12:54

• Changed state: WontFix

[GoogleCodeExporter]

boost build is very good at solving this problem, but enforcing link-compatible 
compiler switches across all dependencies. my assumption would be that the 
DEBUG macro controls whether iterators have asserts in them or not.

Original comment by arvid.no...@gmail.com on 28 Oct 2013 at 12:54

[GoogleCodeExporter]

This is a very old thread (almost 6 months), but since there's still activity 
I'll respond.  I believe my problem was indeed the build switches I was using 
to build / link the libraries.  I believe there is a mismatch between the 
default build switches that boost has configured for the 1.53.0 library source 
and the switches configured for the libtorrent 0.16.9 sources, or there is was 
simply a mismatch between the switches I was using to build my project and 
those used to build libtorrent.  In any case, I had to force both the boost 
libraries and libtorrent builds to match the switches I was using for my 
project before I could get a clean build that would link statically.

Original comment by r...@thepit.org on 29 Oct 2013 at 10:51