search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.05k stars 1.08k forks source link

[YSQL][Unit Test] heap-use-after-free in CDCSDKYsqlTest.TestCreateStreamAfterSetCheckpointMax #14894

Open def- opened 2 years ago

def- commented 2 years ago

Jira Link: DB-4171

Description

On 2.17.1.0-b162 There is a heap-use-after-free in CDCSDKYsqlTest.TestCreateStreamAfterSetCheckpointMax

==7964==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200017b9d0 at pc 0x7f839acd1c80 bp 0x7f8351f755a0 sp 0x7f8351f75598
READ of size 8 at 0x60200017b9d0 thread T71 (TabletServer_re)
    #0 0x7f839acd1c7f in std::unique_ptr<yb::client::YBClient::Data, std::default_delete<yb::client::YBClient::Data>>::operator->[abi:v15003]() const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__memory/unique_ptr.h:274:19
    #1 0x7f839acd1c7f in yb::client::YBClient::callback_threadpool() ${BUILD_ROOT}/../../src/yb/client/client.cc:1891:10
    #2 0x7f839ac8ac83 in yb::client::internal::Batcher::RunCallback() ${BUILD_ROOT}/../../src/yb/client/batcher.cc:209:17
    #3 0x7f839ac8a6c6 in yb::client::internal::Batcher::FlushFinished() ${BUILD_ROOT}/../../src/yb/client/batcher.cc:198:3
    #4 0x7f839ac92cb7 in yb::client::internal::Batcher::Flushed(boost::iterator_range<std::__wrap_iter<yb::client::internal::InFlightOp*>> const&, yb::Status const&, yb::client::internal::FlushExtraResult) ${BUILD_ROOT}/../../src/yb/client/batcher.cc:643:5
    #5 0x7f839ac5bba4 in yb::client::internal::AsyncRpc::Finished(yb::Status const&) ${BUILD_ROOT}/../../src/yb/client/async_rpc.cc:246:15
    #6 0x7f839572c2a8 in yb::rpc::RpcRetrier::DoRetry(yb::rpc::RpcCommand*, yb::Status const&) ${BUILD_ROOT}/../../src/yb/rpc/rpc.cc:237:10
    #7 0x7f8395737411 in std::__bind_return<void (yb::rpc::RpcRetrier::*)(yb::rpc::RpcCommand*, yb::Status const&), std::tuple<yb::rpc::RpcRetrier*, yb::rpc::RpcCommand*, std::placeholders::__ph<1>>, std::tuple<yb::Status const&>, __is_valid_bind_return<void (yb::rpc::RpcRetrier::*)(yb::rpc::RpcCommand*, yb::Status const&), std::tuple<yb::rpc::RpcRetrier*, yb::rpc::RpcCommand*, std::placeholders::__ph<1>>, std::tuple<yb::Status const&>>::value>::type std::__bind<void (yb::rpc::RpcRetrier::*)(yb::rpc::RpcCommand*, yb::Status const&), yb::rpc::RpcRetrier*, yb::rpc::RpcCommand*&, std::placeholders::__ph<1> const&>::operator()[abi:v15003]<yb::Status const&>(yb::Status const&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__functional/bind.h:295:20
    #8 0x7f8395737411 in boost::detail::function::void_function_obj_invoker1<std::__bind<void (yb::rpc::RpcRetrier::*)(yb::rpc::RpcCommand*, yb::Status const&), yb::rpc::RpcRetrier*, yb::rpc::RpcCommand*&, std::placeholders::__ph<1> const&>, void, yb::Status const&>::invoke(boost::detail::function::function_buffer&, yb::Status const&) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/include/boost/function/function_template.hpp:158:11
    #9 0x7f83a2a7ddb1 in boost::function1<void, yb::Status const&>::operator()(yb::Status const&) const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/include/boost/function/function_template.hpp:763:14
    #10 0x7f8395700943 in yb::rpc::DelayedTask::AbortTask(yb::Status const&) ${BUILD_ROOT}/../../src/yb/rpc/reactor.cc:863:5
    #11 0x7f83956f3a9a in yb::rpc::ReactorTask::Abort(yb::Status const&) ${BUILD_ROOT}/../../src/yb/rpc/reactor.cc:765:5
    #12 0x7f83956f3a9a in yb::rpc::Reactor::ShutdownInternal() ${BUILD_ROOT}/../../src/yb/rpc/reactor.cc:249:11
    #13 0x7f83956f7b7d in yb::rpc::Reactor::AsyncHandler(ev::async&, int) ${BUILD_ROOT}/../../src/yb/rpc/reactor.cc:367:5
    #14 0x7f839126ec4a in ev_invoke_pending (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/common/lib/libev.so.4+0x8c4a)
    #15 0x7f839126f99d in ev_run (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/common/lib/libev.so.4+0x999d)
    #16 0x7f83956f1e9d in ev::loop_ref::run(int) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/common/include/ev++.h:211:7
    #17 0x7f83956f1e9d in yb::rpc::Reactor::RunThread() ${BUILD_ROOT}/../../src/yb/rpc/reactor.cc:498:9
    #18 0x7f83944cf162 in std::function<void ()>::operator()() const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__functional/function.h:1197:12
    #19 0x7f83944cf162 in yb::Thread::SuperviseThread(void*) ${BUILD_ROOT}/../../src/yb/util/thread.cc:800:3
    #20 0x7f838f7701ce in start_thread (/lib64/libpthread.so.0+0x81ce) (BuildId: e7ab466bac9c591a41ef266c4ad26d3811b399fa)
    #21 0x7f838f1c3dd2 in clone (/lib64/libc.so.6+0x39dd2) (BuildId: 64aa558dcdda2d8b0d7b04cef33ddbb2d9d8b8b4)

0x60200017b9d0 is located 0 bytes inside of 8-byte region [0x60200017b9d0,0x60200017b9d8)
freed by thread T0 here:
    #0 0x55b9ae41d21d in operator delete(void*) /opt/yb-build/llvm/yb-llvm-v15.0.3-yb-1-1667030060-0b8d1183-almalinux8-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:152:3
    #1 0x7f839ac542ef in yb::AtomicUniquePtr<yb::client::YBClient>::~AtomicUniquePtr() ${BUILD_ROOT}/../../src/yb/util/atomic.h:357:5
    #2 0x7f839ac542ef in yb::client::AsyncClientInitialiser::~AsyncClientInitialiser() ${BUILD_ROOT}/../../src/yb/client/async_initializer.cc:69:1
    #3 0x7f83a18c735d in std::default_delete<yb::client::AsyncClientInitialiser>::operator()[abi:v15003](yb::client::AsyncClientInitialiser*) const /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__memory/unique_ptr.h:48:5
    #4 0x7f83a18c735d in std::unique_ptr<yb::client::AsyncClientInitialiser, std::default_delete<yb::client::AsyncClientInitialiser>>::reset[abi:v15003](yb::client::AsyncClientInitialiser*) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__memory/unique_ptr.h:305:7
    #5 0x7f83a18c735d in std::unique_ptr<yb::client::AsyncClientInitialiser, std::default_delete<yb::client::AsyncClientInitialiser>>::operator=[abi:v15003](std::nullptr_t) /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20221029203216-efbc29b551-almalinux8-x86_64-clang15/installed/asan/libcxx/include/c++/v1/__memory/unique_ptr.h:263:5
    #6 0x7f83a18c735d in yb::cdc::CDCServiceImpl::Shutdown() ${BUILD_ROOT}/../../ent/src/yb/cdc/cdc_service.cc:3415:31
[...]

Full logs: https://gist.github.com/def-/1eac6bde66aae461d465d3b39b67cb11 https://detective.dev.yugabyte.com/?commits=db2b77a403465890752bac157aef819be36ba2d3

def- commented 2 years ago

Also in CDCSDKYsqlTest.TestMultpleStreamOnSameTablet: https://gist.github.com/def-/6af1f8c555201097587382f9f23e07b0 Probably the same issue.