search

oatpp / oatpp

🌱Light and powerful C++ web framework for highly scalable and resource-efficient web application. It's zero-dependency and easy-portable.
https://oatpp.io/
Apache License 2.0
7.82k stars 1.3k forks source link

Thread Sanitizer Warning: IOEventWorker Data Race #433

Open xanderdunn opened 3 years ago

xanderdunn commented 3 years ago

I'm running the async API to make about 700 websocket connections, and I'm seeing these thread sanitizer warnings over and over many hundreds of times in the first minute of execution:

==================
==================
WARNING: ThreadSanitizer: data race (pid=10210)
  Read of size 8 at 0x7ba000000140 by thread T39:
    #0 eventfd_write /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:6353 (libtsan.so.0+0x466bd)
    #1 oatpp::async::worker::IOEventWorker::triggerWakeup() /home/xander/dev/vcpkg/buildtrees/oatpp/src/1.2.5-c679c9d953.clean/src/oatpp/core/async/worker/IOEventWorker_epoll.cpp:84 (DataFeed+0x3bce46)

  Previous write of size 8 at 0x7ba000000140 by thread T63:
    #0 eventfd /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libsanitizer/tsan/tsan_interceptors.cc:1544 (libtsan.so.0+0x300ba)
    #1 oatpp::async::worker::IOEventWorker::initEventQueue() /home/xander/dev/vcpkg/buildtrees/oatpp/src/1.2.5-c679c9d953.clean/src/oatpp/core/async/worker/IOEventWorker_epoll.cpp:57 (DataFeed+0x3bcb72)

  Location is file descriptor 20 created by thread T63 at:
    #0 eventfd /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libsanitizer/tsan/tsan_interceptors.cc:1544 (libtsan.so.0+0x300ba)
    #1 oatpp::async::worker::IOEventWorker::initEventQueue() /home/xander/dev/vcpkg/buildtrees/oatpp/src/1.2.5-c679c9d953.clean/src/oatpp/core/async/worker/IOEventWorker_epoll.cpp:57 (DataFeed+0x3bcb72)

  Thread T39 (tid=10250, running) created by main thread at:
    #0 pthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libsanitizer/tsan/tsan_interceptors.cc:964 (libtsan.so.0+0x2ddfc)
    #1 __gthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/build/build-cc-gcc-final/x86_64-conda-linux-gnu/libstdc++-v3/include/x86_64-conda-linux-gnu/bits/gthr-default.h:676 (libstdc++.so.6+0xc91f8)
    #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:135 (libstdc++.so.6+0xc91f8)
    #3 void std::allocator_traits<std::allocator<oatpp::async::Executor> >::construct<oatpp::async::Executor, int&, int&, int&>(std::allocator<oatpp::async::Executor>&, oatpp::async::Executor*, int&, int&, int&) <null> (DataFeed+0x2c4378)
    #4 std::_Sp_counted_ptr_inplace<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<int&, int&, int&>(std::allocator<oatpp::async::Executor>, int&, int&, int&) <null> (DataFeed+0x2aa87b)
    #5 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, int&, int&, int&>(oatpp::async::Executor*&, std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x28dc15)
    #6 std::__shared_ptr<oatpp::async::Executor, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x265177)
    #7 std::shared_ptr<oatpp::async::Executor>::shared_ptr<std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x23dda8)
    #8 std::shared_ptr<oatpp::async::Executor> std::allocate_shared<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::allocator<oatpp::async::Executor> const&, int&, int&, int&) <null> (DataFeed+0x2055ac)
    #9 std::shared_ptr<oatpp::async::Executor> std::make_shared<oatpp::async::Executor, int&, int&, int&>(int&, int&, int&) <null> (DataFeed+0x1d2ce8)
    #10 mynamespace::MyWebsocketClient::start_connections() /home/xander/dev/my_model/mydata/mydata/MyWebsocketClient.h:467 (DataFeed+0x1aa28f)
    #11 main /home/xander/dev/my_model/mydata/mydata/DataFeed.cc:54 (DataFeed+0xc61f4)

  Thread T63 (tid=10274, running) created by main thread at:
    #0 pthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libsanitizer/tsan/tsan_interceptors.cc:964 (libtsan.so.0+0x2ddfc)
    #1 __gthread_create /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/build/build-cc-gcc-final/x86_64-conda-linux-gnu/libstdc++-v3/include/x86_64-conda-linux-gnu/bits/gthr-default.h:676 (libstdc++.so.6+0xc91f8)
    #2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /home/conda/feedstock_root/build_artifacts/ctng-compilers_1618239181388/work/.build/x86_64-conda-linux-gnu/src/gcc/libstdc++-v3/src/c++11/thread.cc:135 (libstdc++.so.6+0xc91f8)
    #3 void std::allocator_traits<std::allocator<oatpp::async::Executor> >::construct<oatpp::async::Executor, int&, int&, int&>(std::allocator<oatpp::async::Executor>&, oatpp::async::Executor*, int&, int&, int&) <null> (DataFeed+0x2c4378)
    #4 std::_Sp_counted_ptr_inplace<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<int&, int&, int&>(std::allocator<oatpp::async::Executor>, int&, int&, int&) <null> (DataFeed+0x2aa87b)
    #5 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, int&, int&, int&>(oatpp::async::Executor*&, std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x28dc15)
    #6 std::__shared_ptr<oatpp::async::Executor, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x265177)
    #7 std::shared_ptr<oatpp::async::Executor>::shared_ptr<std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::_Sp_alloc_shared_tag<std::allocator<oatpp::async::Executor> >, int&, int&, int&) <null> (DataFeed+0x23dda8)
    #8 std::shared_ptr<oatpp::async::Executor> std::allocate_shared<oatpp::async::Executor, std::allocator<oatpp::async::Executor>, int&, int&, int&>(std::allocator<oatpp::async::Executor> const&, int&, int&, int&) <null> (DataFeed+0x2055ac)
    #9 std::shared_ptr<oatpp::async::Executor> std::make_shared<oatpp::async::Executor, int&, int&, int&>(int&, int&, int&) <null> (DataFeed+0x1d2ce8)
    #10 mynamespace::MyWebsocketClient::start_connections() /home/xander/dev/my_model/mydata/mydata/MyWebsocketClient.h:467 (DataFeed+0x1aa28f)
    #11 main /home/xander/dev/my_model/mydata/mydata/DataFeed.cc:54 (DataFeed+0xc61f4)

SUMMARY: ThreadSanitizer: data race /home/xander/dev/vcpkg/buildtrees/oatpp/src/1.2.5-c679c9d953.clean/src/oatpp/core/async/worker/IOEventWorker_epoll.cpp:84 in oatpp::async::worker::IOEventWorker::triggerWakeup()

I did not see these thread sanitizer warnings a short time ago. I recently upgraded from 1.2.0 to 1.2.5. Is this new as of 1.2.5?

lganzzzo commented 3 years ago

Hey @xanderdunn ,

Thanks for sharing this log. We have to take a closer look at this.

falko-strenzke commented 2 years ago

This but seems to show even when running the unit tests under valgrind (memcheck): ( commit 2f975359c55d6eb37bc0aef1afdc2abfcf031552 )

==187850== Memcheck, a memory error detector
==187850== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==187850== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==187850== Command: ./test/oatppAllTests
==187850== 
[...]
==187850== Thread 23:
==187850== Conditional jump or move depends on uninitialised value(s)
==187850==    at 0x4C254EB: oatpp::async::worker::IOEventWorker::waitEvents() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C23DF9: oatpp::async::worker::IOEventWorker::run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24996: void std::__invoke_impl<void, void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(std::__invoke_memfun_deref, void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C248F4: std::__invoke_result<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>::type std::__invoke<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24864: void std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C2481D: std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::operator()() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24801: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> > >::_M_run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4F128F3: execute_native_thread_routine (thread.cc:82)
==187850==    by 0x4E11EA6: start_thread (pthread_create.c:477)
==187850==    by 0x52A7DEE: clone (clone.S:95)
==187850== 
==187850== Conditional jump or move depends on uninitialised value(s)
==187850==    at 0x4C25501: oatpp::async::worker::IOEventWorker::waitEvents() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C23DF9: oatpp::async::worker::IOEventWorker::run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24996: void std::__invoke_impl<void, void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(std::__invoke_memfun_deref, void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C248F4: std::__invoke_result<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>::type std::__invoke<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24864: void std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C2481D: std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::operator()() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24801: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> > >::_M_run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4F128F3: execute_native_thread_routine (thread.cc:82)
==187850==    by 0x4E11EA6: start_thread (pthread_create.c:477)
==187850==    by 0x52A7DEE: clone (clone.S:95)
==187850== 
==187850== Conditional jump or move depends on uninitialised value(s)
==187850==    at 0x4C25517: oatpp::async::worker::IOEventWorker::waitEvents() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C23DF9: oatpp::async::worker::IOEventWorker::run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24996: void std::__invoke_impl<void, void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(std::__invoke_memfun_deref, void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C248F4: std::__invoke_result<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>::type std::__invoke<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*>(void (oatpp::async::worker::IOEventWorker::*&&)(), oatpp::async::worker::IOEventWorker*&&) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24864: void std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C2481D: std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> >::operator()() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4C24801: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (oatpp::async::worker::IOEventWorker::*)(), oatpp::async::worker::IOEventWorker*> > >::_M_run() (in /home/falko/lib/oat++/oatpp/src/liboatpp.so)
==187850==    by 0x4F128F3: execute_native_thread_routine (thread.cc:82)
==187850==    by 0x4E11EA6: start_thread (pthread_create.c:477)
==187850==    by 0x52A7DEE: clone (clone.S:95)
==187850==