search

pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
944 stars 410 forks source link

TSAN check failed - data race in SegmentReader #7138

Open JaySon-Huang opened 1 year ago

JaySon-Huang commented 1 year ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

[2023-03-21T18:40:26.603Z] ==================
[2023-03-21T18:40:26.603Z] WARNING: ThreadSanitizer: data race (pid=3912)
[2023-03-21T18:40:26.603Z]   Write of size 8 at 0x7b5c000e02e0 by thread T9:
[2023-03-21T18:40:26.603Z]     #0 DB::DM::DMFileReader::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:443:111 (gtests_dbms+0xb352654)
[2023-03-21T18:40:26.603Z]     #1 DB::DM::DMFileBlockInputStream::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.h:64:23 (gtests_dbms+0xb349fd7)
[2023-03-21T18:40:26.603Z]     #2 DB::DM::ConcatSkippableBlockInputStream<false>::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SkippableBlockInputStream.h:175:38 (gtests_dbms+0xb2804f2)
[2023-03-21T18:40:26.603Z]     #3 DB::DM::readBlock(std::__1::shared_ptr<DB::DM::SkippableBlockInputStream>&, std::__1::shared_ptr<DB::DM::SkippableBlockInputStream>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadUtil.cpp:33:26 (gtests_dbms+0xb1fcfbf)
[2023-03-21T18:40:26.603Z]     #4 DB::DM::BitmapFilterBlockInputStream::readImpl(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/BitmapFilter/BitmapFilterBlockInputStream.cpp:40:32 (gtests_dbms+0xb2eb366)
[2023-03-21T18:40:26.603Z]     #5 DB::DM::BitmapFilterBlockInputStream::readImpl() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/BitmapFilter/BitmapFilterBlockInputStream.h:46:16 (gtests_dbms+0xb2ef426)
[2023-03-21T18:40:26.603Z]     #6 DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/DataStreams/IProfilingBlockInputStream.cpp:75:19 (gtests_dbms+0xc7d8359)
[2023-03-21T18:40:26.603Z]     #7 DB::IProfilingBlockInputStream::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/DataStreams/IProfilingBlockInputStream.cpp:43:12 (gtests_dbms+0xc7d7f93)
[2023-03-21T18:40:26.603Z]     #8 DB::DM::SegmentReadTaskPool::readOneBlock(std::__1::shared_ptr<DB::IBlockInputStream>&, std::__1::shared_ptr<DB::DM::Segment> const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:244:26 (gtests_dbms+0xb2ab6da)
[2023-03-21T18:40:26.603Z]     #9 DB::DM::MergedTask::readOneBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:82:19 (gtests_dbms+0xb4081ca)
[2023-03-21T18:40:26.603Z]     #10 DB::DM::MergedTask::readBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:32:12 (gtests_dbms+0xb407d21)
[2023-03-21T18:40:26.603Z]     #11 DB::DM::SegmentReader::readSegments() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:102:39 (gtests_dbms+0xb4140fc)
[2023-03-21T18:40:26.603Z]     #12 DB::DM::SegmentReader::run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:153:13 (gtests_dbms+0xb413917)
[2023-03-21T18:40:26.603Z]     #13 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.603Z]     #14 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.603Z]     #15 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.603Z] 
[2023-03-21T18:40:26.603Z]   Previous read of size 8 at 0x7b5c000e02e0 by thread T7 (mutexes: write M1058627387722546432):
[2023-03-21T18:40:26.603Z]     #0 DB::DM::DMFileReader::addCachedPacks(long, unsigned long, unsigned long, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:714:9 (gtests_dbms+0xb356513)
[2023-03-21T18:40:26.603Z]     #1 DB::DM::DMFileReaderPool::set(DB::DM::DMFileReader&, long, unsigned long, unsigned long, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/ColumnSharingCache.cpp:60:12 (gtests_dbms+0xb3f4664)
[2023-03-21T18:40:26.604Z]     #2 DB::DM::DMFileReader::readColumn(DB::DM::ColumnDefine&, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:704:38 (gtests_dbms+0xb355abf)
[2023-03-21T18:40:26.604Z]     #3 DB::DM::DMFileReader::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:612:25 (gtests_dbms+0xb353c63)
[2023-03-21T18:40:26.604Z]     #4 DB::DM::DMFileBlockInputStream::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.h:64:23 (gtests_dbms+0xb349fd7)
[2023-03-21T18:40:26.604Z]     #5 DB::DM::ConcatSkippableBlockInputStream<false>::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SkippableBlockInputStream.h:175:38 (gtests_dbms+0xb2804f2)
[2023-03-21T18:40:26.604Z]     #6 DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false, true>::fillStableBlockIfNeeded() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMerge.h:390:43 (gtests_dbms+0xb28b584)
[2023-03-21T18:40:26.604Z]     #7 void DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false, true>::writeFromStable<false>(std::__1::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMerge.h:476:22 (gtests_dbms+0xb28c372)
[2023-03-21T18:40:26.604Z]     #8 void DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false, true>::next<false, false>(std::__1::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMerge.h:312:17 (gtests_dbms+0xb288f1a)
[2023-03-21T18:40:26.604Z]     #9 DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false, true>::doRead() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMerge.h:263:25 (gtests_dbms+0xb288f1a)
[2023-03-21T18:40:26.604Z]     #10 DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueReader, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false, true>::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMerge.h:187:22 (gtests_dbms+0xb2882eb)
[2023-03-21T18:40:26.604Z]     #11 DB::DM::DMRowKeyFilterBlockInputStream<true>::read() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/RowKeyFilter.h:208:44 (gtests_dbms+0x70b0c71)
[2023-03-21T18:40:26.604Z]     #12 DB::DM::readNextBlock(std::__1::shared_ptr<DB::IBlockInputStream> const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaMergeHelpers.h:258:25 (gtests_dbms+0x6c593ba)
[2023-03-21T18:40:26.604Z]     #13 DB::DM::DMVersionFilterBlockInputStream<0>::initNextBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.h:139:21 (gtests_dbms+0xb1797a6)
[2023-03-21T18:40:26.604Z]     #14 DB::DM::DMVersionFilterBlockInputStream<0>::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.cpp:51:18 (gtests_dbms+0xb1779e9)
[2023-03-21T18:40:26.604Z]     #15 DB::DM::BitmapFilter::set(std::__1::shared_ptr<DB::IBlockInputStream>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/BitmapFilter/BitmapFilter.cpp:32:28 (gtests_dbms+0xb2e6215)
[2023-03-21T18:40:26.604Z]     #16 DB::DM::Segment::buildBitmapFilterNormal(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2488:20 (gtests_dbms+0xb25e37c)
[2023-03-21T18:40:26.604Z]     #17 DB::DM::Segment::buildBitmapFilter(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2462:16 (gtests_dbms+0xb25b4d3)
[2023-03-21T18:40:26.604Z]     #18 DB::DM::Segment::getBitmapFilterInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2822:26 (gtests_dbms+0xb241345)
[2023-03-21T18:40:26.604Z]     #19 DB::DM::Segment::getInputStream(DB::DM::ReadMode const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:690:16 (gtests_dbms+0xb23ea69)
[2023-03-21T18:40:26.604Z]     #20 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:163:26 (gtests_dbms+0xb2aa479)
[2023-03-21T18:40:26.604Z]     #21 DB::DM::MergedTask::initOnce() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:50:24 (gtests_dbms+0xb407e92)
[2023-03-21T18:40:26.604Z]     #22 DB::DM::MergedTask::readBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xb407d19)
[2023-03-21T18:40:26.604Z]     #23 DB::DM::SegmentReader::readSegments() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:102:39 (gtests_dbms+0xb4140fc)
[2023-03-21T18:40:26.604Z]     #24 DB::DM::SegmentReader::run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:153:13 (gtests_dbms+0xb413917)
[2023-03-21T18:40:26.605Z]     #25 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.605Z]     #26 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.605Z]     #27 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.605Z] 
[2023-03-21T18:40:26.605Z]   Location is heap block of size 840 at 0x7b5c000e0000 allocated by thread T9:
[2023-03-21T18:40:26.605Z]     #0 operator new(unsigned long) /llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64:3 (gtests_dbms+0x2798f18)
[2023-03-21T18:40:26.605Z]     #1 void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) /usr/local/bin/../include/c++/v1/new:235:10 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #2 std::__1::__libcpp_allocate(unsigned long, unsigned long) /usr/local/bin/../include/c++/v1/new:261:10 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #3 std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > >::allocate(unsigned long) /usr/local/bin/../include/c++/v1/__memory/allocator.h:108:38 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #4 std::__1::allocator_traits<std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > > >::allocate(std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > >&, unsigned long) /usr/local/bin/../include/c++/v1/__memory/allocator_traits.h:262:20 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #5 std::__1::__allocation_guard<std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > > >::__allocation_guard<std::__1::allocator<DB::DM::DMFileBlockInputStream> >(std::__1::allocator<DB::DM::DMFileBlockInputStream>, unsigned long) /usr/local/bin/../include/c++/v1/__memory/allocation_guard.h:57:18 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #6 std::__1::shared_ptr<DB::DM::DMFileBlockInputStream> std::__1::allocate_shared<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream>, DB::DM::DMFileReader, bool&, void>(std::__1::allocator<DB::DM::DMFileBlockInputStream> const&, DB::DM::DMFileReader&&, bool&) /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:1105:48 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #7 std::__1::shared_ptr<DB::DM::DMFileBlockInputStream> std::__1::make_shared<DB::DM::DMFileBlockInputStream, DB::DM::DMFileReader, bool&, void>(DB::DM::DMFileReader&&, bool&) /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:1115:12 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #8 DB::DM::DMFileBlockInputStreamBuilder::build(std::__1::shared_ptr<DB::DM::DMFile> const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::ScanContext> const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.cpp:79:12 (gtests_dbms+0xb344de6)
[2023-03-21T18:40:26.605Z]     #9 DB::DM::StableValueSpace::Snapshot::getInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long, bool, bool, bool, std::__1::vector<std::__1::shared_ptr<std::__1::set<unsigned long, std::__1::less<unsigned long>, std::__1::allocator<unsigned long> > >, std::__1::allocator<std::__1::shared_ptr<std::__1::set<unsigned long, std::__1::less<unsigned long>, std::__1::allocator<unsigned long> > > > > const&, bool) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/StableValueSpace.cpp:445:35 (gtests_dbms+0xb2bb016)
[2023-03-21T18:40:26.605Z]     #10 DB::DM::Segment::getBitmapFilterInputStream(std::__1::shared_ptr<DB::DM::BitmapFilter>&&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2675:72 (gtests_dbms+0xb25e9ae)
[2023-03-21T18:40:26.605Z]     #11 DB::DM::Segment::getBitmapFilterInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2844:12 (gtests_dbms+0xb2414c1)
[2023-03-21T18:40:26.605Z]     #12 DB::DM::Segment::getInputStream(DB::DM::ReadMode const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:690:16 (gtests_dbms+0xb23ea69)
[2023-03-21T18:40:26.605Z]     #13 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:163:26 (gtests_dbms+0xb2aa479)
[2023-03-21T18:40:26.605Z]     #14 DB::DM::MergedTask::initOnce() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:50:24 (gtests_dbms+0xb407e92)
[2023-03-21T18:40:26.606Z]     #15 DB::DM::MergedTask::readBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xb407d19)
[2023-03-21T18:40:26.606Z]     #16 DB::DM::SegmentReader::readSegments() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:102:39 (gtests_dbms+0xb4140fc)
[2023-03-21T18:40:26.606Z]     #17 DB::DM::SegmentReader::run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:153:13 (gtests_dbms+0xb413917)
[2023-03-21T18:40:26.606Z]     #18 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.606Z]     #19 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.606Z]     #20 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xb415963)
[2023-03-21T18:40:26.606Z] 
[2023-03-21T18:40:26.606Z]   Mutex M1058627387722546432 is already destroyed.
[2023-03-21T18:40:26.606Z] 
[2023-03-21T18:40:26.606Z]   Thread T9 'SegmentReader' (tid=4067, running) created by main thread at:
[2023-03-21T18:40:26.606Z]     #0 pthread_create /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x272433d)
[2023-03-21T18:40:26.606Z]     #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xb4139f5)
[2023-03-21T18:40:26.606Z]     #2 std::__1::thread::thread<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xb4139f5)
[2023-03-21T18:40:26.606Z]     #3 DB::DM::SegmentReader::SegmentReader(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:37:13 (gtests_dbms+0xb413808)
[2023-03-21T18:40:26.606Z]     #4 std::__1::__unique_if<DB::DM::SegmentReader>::__unique_single std::__1::make_unique<DB::DM::SegmentReader, DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&>(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xb40c1a3)
[2023-03-21T18:40:26.606Z]     #5 DB::DM::SegmentReaderPool::SegmentReaderPool(int, std::__1::vector<int, std::__1::allocator<int> > const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:180:27 (gtests_dbms+0xb40c1a3)
[2023-03-21T18:40:26.606Z]     #6 std::__1::__unique_if<DB::DM::SegmentReaderPool>::__unique_single std::__1::make_unique<DB::DM::SegmentReaderPool, unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&>(unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xb40cd38)
[2023-03-21T18:40:26.606Z]     #7 DB::DM::SegmentReaderPoolManager::init(unsigned int, double) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:220:32 (gtests_dbms+0xb40cd38)
[2023-03-21T18:40:26.606Z]     #8 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:75:50 (gtests_dbms+0x76ae1b8)
[2023-03-21T18:40:26.606Z] 
[2023-03-21T18:40:26.606Z]   Thread T7 'SegmentReader' (tid=4065, running) created by main thread at:
[2023-03-21T18:40:26.606Z]     #0 pthread_create /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x272433d)
[2023-03-21T18:40:26.606Z]     #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xb4139f5)
[2023-03-21T18:40:26.606Z]     #2 std::__1::thread::thread<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xb4139f5)
[2023-03-21T18:40:26.606Z]     #3 DB::DM::SegmentReader::SegmentReader(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:37:13 (gtests_dbms+0xb413808)
[2023-03-21T18:40:26.606Z]     #4 std::__1::__unique_if<DB::DM::SegmentReader>::__unique_single std::__1::make_unique<DB::DM::SegmentReader, DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&>(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xb40c1a3)
[2023-03-21T18:40:26.606Z]     #5 DB::DM::SegmentReaderPool::SegmentReaderPool(int, std::__1::vector<int, std::__1::allocator<int> > const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:180:27 (gtests_dbms+0xb40c1a3)
[2023-03-21T18:40:26.606Z]     #6 std::__1::__unique_if<DB::DM::SegmentReaderPool>::__unique_single std::__1::make_unique<DB::DM::SegmentReaderPool, unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&>(unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xb40cd38)
[2023-03-21T18:40:26.606Z]     #7 DB::DM::SegmentReaderPoolManager::init(unsigned int, double) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:220:32 (gtests_dbms+0xb40cd38)
[2023-03-21T18:40:26.606Z]     #8 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:75:50 (gtests_dbms+0x76ae1b8)
[2023-03-21T18:40:26.606Z] 
[2023-03-21T18:40:26.607Z] SUMMARY: ThreadSanitizer: data race /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:443:111 in DB::DM::DMFileReader::read()
[2023-03-21T18:40:26.607Z] ==================

4. What is your TiFlash version? (Required)

master

JaySon-Huang commented 1 year ago

/assign @JinheLin