search

pistacheio / pistache

A high-performance REST toolkit written in C++
https://pistacheio.github.io/pistache/
Apache License 2.0
3.2k stars 699 forks source link

Data race in Pistache::Rest::Router::route #1238

Closed tyler92 closed 1 month ago

tyler92 commented 2 months ago

A data race occurs when several HTTP requests are handled in parallel. Multiple threads write to the same std::unordered_map (routes) without synchronization. This could lead to memory corruption or writing to an invalid address.

WARNING: ThreadSanitizer: data race (pid=799942)
  Write of size 8 at 0x7b2c00000020 by thread T3 (mutexes: write M0):
    #0 std::_Hashtable<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>>::_M_insert_bucket_begin(unsigned long, std::__detail::_Hash_node<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, true>*) /usr/include/c++/13/bits/hashtable.h:2009:27 (fuzz_server+0x2f10cc) 
    #1 std::_Hashtable<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>>::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, true>*, unsigned long) /usr/include/c++/13/bits/hashtable.h:2171:7 (fuzz_server+0x2f005b) 
    #2 std::__detail::_Map_base<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](Pistache::Http::Method&&) /usr/include/c++/13/bits/hashtable_policy.h:848:9 (fuzz_server+0x2f3aed) 
    #3 std::unordered_map<Pistache::Http::Method, Pistache::Rest::SegmentTreeNode, std::hash<Pistache::Http::Method>, std::equal_to<Pistache::Http::Method>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>>::operator[](Pistache::Http::Method&&) /usr/include/c++/13/bits/unordered_map.h:991:16 (fuzz_server+0x2dc825) 
    #4 Pistache::Rest::Router::route(Pistache::Http::Request const&, Pistache::Http::ResponseWriter) pistacheio/pistache/src/server/router.cc:522:32 (fuzz_server+0x2d745b) 
    #5 Pistache::Rest::Private::RouterHandler::onRequest(Pistache::Http::Request const&, Pistache::Http::ResponseWriter) pistacheio/pistache/src/server/router.cc:373:21 (fuzz_server+0x2d715a) 
    #6 Pistache::Http::Handler::onInput(char const*, unsigned long, std::shared_ptr<Pistache::Tcp::Peer> const&) pistacheio/pistache/src/common/http.cc:1283:17 (fuzz_server+0x204200) 
    #7 Pistache::Tcp::Transport::handleIncoming(std::shared_ptr<Pistache::Tcp::Peer> const&) pistacheio/pistache/src/common/transport.cc:345:27 (fuzz_server+0x28f1ed) 
    #8 Pistache::Tcp::Transport::onReady(Pistache::Aio::FdSet const&) pistacheio/pistache/src/common/transport.cc:214:21 (fuzz_server+0x28e640) 
    #9 Pistache::Http::TransportImpl::onReady(Pistache::Aio::FdSet const&) pistacheio/pistache/src/server/endpoint.cc:158:15 (fuzz_server+0x2aed33) 
    #10 Pistache::Aio::SyncImpl::handleFds(std::vector<Pistache::Polling::Event, std::allocator<Pistache::Polling::Event>>) const pistacheio/pistache/src/common/reactor.cc:294:34 (fuzz_server+0x32c747) 
    #11 Pistache::Aio::SyncImpl::runOnce() pistacheio/pistache/src/common/reactor.cc:246:25 (fuzz_server+0x32a546) 
    #12 Pistache::Aio::SyncImpl::run() pistacheio/pistache/src/common/reactor.cc:263:17 (fuzz_server+0x32a67c) 
    #13 Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()::operator()() const pistacheio/pistache/src/common/reactor.cc:647:27 (fuzz_server+0x3374e1) 
    #14 void std::__invoke_impl<void, Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>(std::__invoke_other, Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()&&) /usr/include/c++/13/bits/invoke.h:61:14 (fuzz_server+0x337405) 
    #15 std::__invoke_result<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>::type std::__invoke<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>(Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()&&) /usr/include/c++/13/bits/invoke.h:96:14 (fuzz_server+0x337375) 
    #16 void std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/13/bits/std_thread.h:292:13 (fuzz_server+0x33732d) 
    #17 std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>::operator()() /usr/include/c++/13/bits/std_thread.h:299:11 (fuzz_server+0x3372d5) 
    #18 std::thread::_State_impl<std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>>::_M_run() /usr/include/c++/13/bits/std_thread.h:244:13 (fuzz_server+0x337199) 
    #19 <null> <null> (libstdc++.so.6+0xe62b2) 

  Previous read of size 8 at 0x7b2c00000020 by thread T2 (mutexes: write M1):
    #0 std::_Hashtable<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>>::_M_find_before_node(unsigned long, Pistache::Http::Method const&, unsigned long) const /usr/include/c++/13/bits/hashtable.h:1943:63 (fuzz_server+0x2f0354) 
    #1 std::_Hashtable<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>>::_M_find_node(unsigned long, Pistache::Http::Method const&, unsigned long) const /usr/include/c++/13/bits/hashtable.h:815:31 (fuzz_server+0x2efd65) 
    #2 std::__detail::_Map_base<Pistache::Http::Method, std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>, std::__detail::_Select1st, std::equal_to<Pistache::Http::Method>, std::hash<Pistache::Http::Method>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](Pistache::Http::Method&&) /usr/include/c++/13/bits/hashtable_policy.h:838:30 (fuzz_server+0x2f3a50) 
    #3 std::unordered_map<Pistache::Http::Method, Pistache::Rest::SegmentTreeNode, std::hash<Pistache::Http::Method>, std::equal_to<Pistache::Http::Method>, std::allocator<std::pair<Pistache::Http::Method const, Pistache::Rest::SegmentTreeNode>>>::operator[](Pistache::Http::Method&&) /usr/include/c++/13/bits/unordered_map.h:991:16 (fuzz_server+0x2dc825) 
    #4 Pistache::Rest::Router::route(Pistache::Http::Request const&, Pistache::Http::ResponseWriter) pistacheio/pistache/src/server/router.cc:522:32 (fuzz_server+0x2d745b) 
    #5 Pistache::Rest::Private::RouterHandler::onRequest(Pistache::Http::Request const&, Pistache::Http::ResponseWriter) pistacheio/pistache/src/server/router.cc:373:21 (fuzz_server+0x2d715a) 
    #6 Pistache::Http::Handler::onInput(char const*, unsigned long, std::shared_ptr<Pistache::Tcp::Peer> const&) pistacheio/pistache/src/common/http.cc:1283:17 (fuzz_server+0x204200) 
    #7 Pistache::Tcp::Transport::handleIncoming(std::shared_ptr<Pistache::Tcp::Peer> const&) pistacheio/pistache/src/common/transport.cc:345:27 (fuzz_server+0x28f1ed) 
    #8 Pistache::Tcp::Transport::onReady(Pistache::Aio::FdSet const&) pistacheio/pistache/src/common/transport.cc:214:21 (fuzz_server+0x28e640) 
    #9 Pistache::Http::TransportImpl::onReady(Pistache::Aio::FdSet const&) pistacheio/pistache/src/server/endpoint.cc:158:15 (fuzz_server+0x2aed33) 
    #10 Pistache::Aio::SyncImpl::handleFds(std::vector<Pistache::Polling::Event, std::allocator<Pistache::Polling::Event>>) const pistacheio/pistache/src/common/reactor.cc:294:34 (fuzz_server+0x32c747) 
    #11 Pistache::Aio::SyncImpl::runOnce() pistacheio/pistache/src/common/reactor.cc:246:25 (fuzz_server+0x32a546) 
    #12 Pistache::Aio::SyncImpl::run() pistacheio/pistache/src/common/reactor.cc:263:17 (fuzz_server+0x32a67c) 
    #13 Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()::operator()() const pistacheio/pistache/src/common/reactor.cc:647:27 (fuzz_server+0x3374e1) 
    #14 void std::__invoke_impl<void, Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>(std::__invoke_other, Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()&&) /usr/include/c++/13/bits/invoke.h:61:14 (fuzz_server+0x337405) 
    #15 std::__invoke_result<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>::type std::__invoke<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>(Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()&&) /usr/include/c++/13/bits/invoke.h:96:14 (fuzz_server+0x337375) 
    #16 void std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/include/c++/13/bits/std_thread.h:292:13 (fuzz_server+0x33732d) 
    #17 std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>::operator()() /usr/include/c++/13/bits/std_thread.h:299:11 (fuzz_server+0x3372d5) 
    #18 std::thread::_State_impl<std::thread::_Invoker<std::tuple<Pistache::Aio::AsyncImpl::Worker::run()::'lambda'()>>>::_M_run() /usr/include/c++/13/bits/std_thread.h:244:13 (fuzz_server+0x337199) 
    #19 <null> <null> (libstdc++.so.6+0xe62b2)

Full thread sanitizer report: tsan.log

dgreatwood commented 2 months ago

Perfect, thanks much.

BTW, I will run some more overnight/soak tests before we take this change into master, we may do some more review etc., so it might be a few days.

Also, I’ll look at the other issue, though it’ll be next week sometime before I can.

Thanks again!

On Sat, Aug 31, 2024 at 4:16 AM tyler92 @.***> wrote:

A data race occurs when several HTTP requests are handled in parallel. Multiple threads write to the same std::unordered_map (routes) without synchronization. This could lead to memory corruption or writing to an invalid address.

Thread sanitizer report: tsan.log https://github.com/user-attachments/files/16824481/tsan.log

— Reply to this email directly, view it on GitHub https://github.com/pistacheio/pistache/issues/1238, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFMA25UN65S6RNX44QYOL3ZUGQZTAVCNFSM6AAAAABNNZEJIGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4TQNZZGU2DONA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- NOTICE: This email and its attachments may contain privileged and confidential information, only for the viewing and use of the intended recipient. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, acting upon, or use of the information contained in this email and its attachments is strictly prohibited and that this email and its attachments must be immediately returned to the sender and deleted from your system. If you received this email erroneously, please notify the sender immediately.  Xage Security, Inc. and its affiliates will never request personal information (e.g., passwords, Social Security numbers) via email.  Report suspicious emails to @. @.>