search

stoneatom / stonedb

StoneDB is an Open-Source MySQL HTAP and MySQL-Native DataBase for OLTP, Real-Time Analytics, a counterpart of MySQLHeatWave. (https://stonedb.io)
https://stonedb.io/
GNU General Public License v2.0
862 stars 140 forks source link

feature: Cache disk files for intermediate results of aggregate operations to avoid OOM #949

Closed adofsauron closed 1 year ago

adofsauron commented 1 year ago

Abstract:

TIANMU Engine - Aggregate operation intermediate results cache disk files to avoid OOM- requirements analysis

Related ISSUE: https://github.com/stoneatom/stonedb/issues/21

Note of context:

The results of the current aggregate operation are cached in the in-memory HASH. Once the amount of data exceeds the RAM, the OOM occurs.

If the data volume exceeds RAM, convert it to disk file storage to avoid OOM

Functional Requirements:

1. The HASH result of the aggregation operation is cached in the disk file and properly participates in the operation

When the intermediate result exceeds RAM

Performance requirements:

1. The compression ratio of the cache file size to the original data content, and the ratio of disk space to memory

Influence compression algorithm

2. Disk cache file write speed and read speed

Standard disk hardware

It affects the speed of aggregation operations and the rules for writing and reading disk cache files

3. The impact on the performance of aggregation operations

Development cycle:

TODO:

adofsauron commented 1 year ago

What are the static and dynamic flows of the aggregate's overall architecture

From the analysis of module layer, it can be divided into several modules

What is the focus of the responsibilities of different modules, and what are the boundaries of interaction between modules

How do classes interact within a single module, and what are the relationships between classes? Pan China? A combination? Aggregation? CRTP(Singular Template Programming)?

adofsauron commented 1 year ago

0 0x000000002d48c6ea in DB::AggregatingTransform::AggregatingTransform (this=0x7f332b252c18, header=..., params_=..., manydata=..., current_variant=0, maxthreads=16,

temporary_data_merge_threads_=16) at ../src/Processors/Transforms/AggregatingTransform.cpp:397

1 0x000000002ca624f7 in std::1::construct_at[abi:v15000]<DB::AggregatingTransform, DB::Block const&, std::__1::shared_ptr&, std::1::shared_ptr&, unsigned long, unsigned long&, unsigned long&, DB::AggregatingTransform>(DB::AggregatingTransform, DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long&&, unsigned long&, unsigned long&) (location=0x7f332b252c18, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16,

__args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16) at ../contrib/llvm-project/libcxx/include/__memory/construct_at.h:35

2 0x000000002ca621bb in std::1::allocator_traits<std::1::allocator >::construct[abi:v15000]<DB::AggregatingTransform, DB::Block const&, std::1::shared_ptr&, std::1::shared_ptr&, unsigned long, unsigned long&, unsigned long&, void, void>(std::1::allocator&, DB::AggregatingTransform*, DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long&&, unsigned long&, unsigned long&) (p=0x7f332b252c18, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16,

__args=@0x7f332b193228: 16) at ../contrib/llvm-project/libcxx/include/__memory/allocator_traits.h:298

3 0x000000002ca61e19 in std::1::shared_ptr_emplace<DB::AggregatingTransform, std::1::allocator >::shared_ptr_emplace[abi:v15000]<DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long, unsigned long&, unsigned long&>(DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long&&, unsigned long&, unsigned long&) (this=0x7f332b252c00, a=..., args=@0x7f332b193228: 16,

__args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16)
at ../contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:292

4 0x000000002ca61b88 in std::1::allocate_shared[abi:v15000]<DB::AggregatingTransform, std::1::allocator, DB::Block const&, std::1::shared_ptr&, std::1::shared_ptr&, unsigned long, unsigned long&, unsigned long&, void>(std::1::allocator const&, DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long&&, unsigned long&, unsigned long&) (__a=...,

__args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16)
at ../contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:953

5 0x000000002ca61a80 in std::1::make_shared[abi:v15000]<DB::AggregatingTransform, DB::Block const&, std::__1::shared_ptr&, std::1::shared_ptr&, unsigned long, unsigned long&, unsigned long&, void>(DB::Block const&, std::1::shared_ptr&, std::__1::shared_ptr&, unsigned long&&, unsigned long&, unsigned long&) (args=@0x7f332b193228: 16, args=@0x7f332b193228: 16, args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16,

__args=@0x7f332b193228: 16, __args=@0x7f332b193228: 16) at ../contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:962

6 0x000000002d6f3c52 in DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6::operator()(DB::Block const&) const (

this=0x7f332b226be0, header=...) at ../src/Processors/QueryPlan/AggregatingStep.cpp:421

7 0x000000002d6f3bbd in std::1::invoke[abi:v15000]<DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6&, DB::Block const&>

(__f=..., __args=...) at ../contrib/llvm-project/libcxx/include/__functional/invoke.h:394

8 0x000000002d6f3b52 in std::1::invoke_void_return_wrapper<std::__1::shared_ptr, false>::call<DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6&, DB::Block const&>(DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6&, DB::Block const&) (args=..., args=...) at ../contrib/llvm-project/libcxx/include/functional/invoke.h:470

9 0x000000002d6f3aed in std::1::function::default_alloc_func<DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6, std::__1::shared_ptr (DB::Block const&)>::operator()[abi:v15000](DB::Block const&) (this=0x7f332b226be0, arg=...)

at ../contrib/llvm-project/libcxx/include/__functional/function.h:235

10 0x000000002d6f3a98 in std::1::function::policy_invoker<std::__1::shared_ptr (DB::Block const&)>::call_impl<std::1::function::default_alloc_func<DB::AggregatingStep::transformPipeline(DB::QueryPipelineBuilder&, DB::BuildQueryPipelineSettings const&)::$_6, std::__1::shared_ptr (DB::Block const&)> >(std::1::function::policy_storage const*, DB::Block const&) (buf=0x7f33a43e9108, args=...) at ../contrib/llvm-project/libcxx/include/__functional/function.h:716

11 0x000000002a1f9a9b in std::1::function::__policy_func<std::__1::shared_ptr (DB::Block const&)>::operator()[abi:v15000](DB::Block const&) const (

this=0x7f33a43e9108, __args=...) at ../contrib/llvm-project/libcxx/include/__functional/function.h:848

12 0x000000002a1f9a2d in std::1::function<std::1::shared_ptr (DB::Block const&)>::operator()(DB::Block const&) const (this=0x7f33a43e9108, __arg=...)

at ../contrib/llvm-project/libcxx/include/__functional/function.h:1187

13 0x000000002a1e3443 in DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2::operator()(DB::Block const&, DB::Pipe::StreamType) const (this=0x7f33a43e8e80, stream_header=...) at ../src/QueryPipeline/Pipe.cpp:668

--Type for more, q to quit, c to continue without paging--

14 0x000000002a1e33e7 in std::1::invoke[abi:v15000]<DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2&, DB::Block const&, DB::Pipe::StreamType> (f=..., args=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main, __args=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main)

at ../contrib/llvm-project/libcxx/include/__functional/invoke.h:394

15 0x000000002a1e33a7 in std::1::invoke_void_return_wrapper<std::__1::shared_ptr, false>::call<DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2&, DB::Block const&, DB::Pipe::StreamType>(DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2&, DB::Block const&, DB::Pipe::StreamType&&) (args=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main, __args=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main,

__args=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main) at ../contrib/llvm-project/libcxx/include/__functional/invoke.h:470

16 0x000000002a1e3342 in std::1::function::default_alloc_func<DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2, std::1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)>::operator()[abi:v15000](DB::Block const&, DB::Pipe::StreamType&&) (this=0x7f33a43e8e80,

__arg=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main, __arg=@0x7f33a43e8b1c: DB::Pipe::StreamType::Main) at ../contrib/llvm-project/libcxx/include/__functional/function.h:235

17 0x000000002a1e32e9 in std::1::function::policy_invoker<std::__1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)>::call_impl<std::1::function::default_alloc_func<DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&)::$_2, std::1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)> >(std::1::function::__policy_storage const*, DB::Block const&, DB::Pipe::StreamType) (buf=0x7f33a43e8e80, args=DB::Pipe::StreamType::Main,

__args=DB::Pipe::StreamType::Main) at ../contrib/llvm-project/libcxx/include/__functional/function.h:716

18 0x000000002a1f58f2 in std::1::function::policy_func<std::__1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)>::operator()[abi:v15000](DB::Block const&, DB::Pipe::StreamType&&) const (this=0x7f33a43e8e80, args=@0x7f33a43e8bbc: DB::Pipe::StreamType::Main, __args=@0x7f33a43e8bbc: DB::Pipe::StreamType::Main)

at ../contrib/llvm-project/libcxx/include/__functional/function.h:848

19 0x000000002a1f5841 in std::1::function<std::1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)>::operator()(DB::Block const&, DB::Pipe::StreamType) const (

this=0x7f33a43e8e80, __arg=DB::Pipe::StreamType::Main, __arg=DB::Pipe::StreamType::Main) at ../contrib/llvm-project/libcxx/include/__functional/function.h:1187

20 0x000000002a1e0b0e in DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)> const&)::$_1::operator()(DB::OutputPort*&, DB::Pipe::StreamType) const (this=0x7f33a43e8dc0, port=@0x7f332b1af240: 0x7f332b236b10, stream_type=DB::Pipe::StreamType::Main) at ../src/QueryPipeline/Pipe.cpp:618

21 0x000000002a1e09df in DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&, DB::Pipe::StreamType)> const&) (this=0x7f332b192ca0,

getter=...) at ../src/QueryPipeline/Pipe.cpp:658

22 0x000000002a1e107e in DB::Pipe::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&) (this=0x7f332b192ca0, getter=...)

at ../src/QueryPipeline/Pipe.cpp:668

23 0x000000002a212b0e in DB::QueryPipelineBuilder::addSimpleTransform(std::1::function<std::1::shared_ptr (DB::Block const&)> const&) (this=0x7f332b192c40,

getter=...) at ../src/QueryPipeline/QueryPipelineBuilder.cpp:128

24 0x000000002d6ef62b in DB::AggregatingStep::transformPipeline (this=0x7f332b193000, pipeline=..., settings=...) at ../src/Processors/QueryPlan/AggregatingStep.cpp:419

25 0x000000002d75f3c7 in DB::ITransformingStep::updatePipeline (this=0x7f332b193000, pipelines=..., settings=...) at ../src/Processors/QueryPlan/ITransformingStep.cpp:48

26 0x000000002d786160 in DB::QueryPlan::buildQueryPipeline (this=0x7f33a43e9a80, optimization_settings=..., build_pipeline_settings=...) at ../src/Processors/QueryPlan/QueryPlan.cpp:187

27 0x000000002b7124d1 in DB::InterpreterSelectWithUnionQuery::execute (this=0x7f339cd04820) at ../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:384

28 0x000000002bbf41b5 in DB::executeQueryImpl (

begin=0x7f332b178000 "select\nc_name,\nc_custkey,\no_orderkey,\no_orderdate,\no_totalprice,\nsum(l_quantity)\nfrom\ncustomer,\norders,\nlineitem\nwhere\nc_custkey = o_custkey\nand o_orderkey = l_orderkey\ngroup by\nc_name,\nc_custkey,\no_o"..., end=0x7f332b17811c "", context=..., internal=false, stage=DB::QueryProcessingStage::Complete, istr=0x0)
at ../src/Interpreters/executeQuery.cpp:715

29 0x000000002bbf0124 in DB::executeQuery (query=..., context=..., internal=false, stage=DB::QueryProcessingStage::Complete) at ../src/Interpreters/executeQuery.cpp:1180

30 0x000000002cfd209b in DB::TCPHandler::runImpl (this=0x7f339ccf5c00) at ../src/Server/TCPHandler.cpp:389

31 0x000000002cfe27e5 in DB::TCPHandler::run (this=0x7f339ccf5c00) at ../src/Server/TCPHandler.cpp:1963

32 0x0000000032066419 in Poco::Net::TCPServerConnection::start (this=0x7f339ccf5c00) at ../base/poco/Net/src/TCPServerConnection.cpp:43

33 0x0000000032066c7b in Poco::Net::TCPServerDispatcher::run (this=0x7f3463f9d600) at ../base/poco/Net/src/TCPServerDispatcher.cpp:115

34 0x00000000322cdf94 in Poco::PooledThread::run (this=0x7f34640e3e80) at ../base/poco/Foundation/src/ThreadPool.cpp:199

35 0x00000000322ca8fa in Poco::(anonymous namespace)::RunnableHolder::run (this=0x7f3464001850) at ../base/poco/Foundation/src/Thread.cpp:55

36 0x00000000322c969c in Poco::ThreadImpl::runnableEntry (pThread=0x7f34640e3eb8) at ../base/poco/Foundation/src/Thread_POSIX.cpp:345

37 0x00007f3464f8a802 in start_thread () from /lib64/libc.so.6

--Type for more, q to quit, c to continue without paging--

38 0x00007f3464f2a450 in clone3 () from /lib64/libc.so.6

adofsauron commented 1 year ago

Why write aggregate data to disk files and what is the purpose of aggregating it into separate pipelines

adofsauron commented 1 year ago

An aggregation is done with store of temporary data on the disk, and they need to be merged in a memory efficient way.

adofsauron commented 1 year ago

The new size fits into the last MemoryChunk, so just alloc the additional size. We can alloc without alignment here, because it only applies to the start of the range, and we don't change it.

adofsauron commented 1 year ago

Begin or expand a contiguous range of memory. 'range_start' is the start of range. If nullptr, a new range is allocated. If there is no space in the current MemoryChunk to expand the range, the entire range is copied to a new, bigger memory MemoryChunk, and the va of 'range_start' is updated. If the optional 'start_alignment' is specified, the start of range is kept aligned to this value.

NOTE This method is usable only for the last allocation made on this Arena. For earlier allocations, see 'realloc' method.

adofsauron commented 1 year ago

Memory pool to append something. For example, short strings. Usage scenario:

adofsauron commented 1 year ago 
    /// If it is claimed that the zero key can not be inserted into the table.
    if constexpr (!Cell::need_zero_value_storage)
        return false;

    if (unlikely(Cell::isZero(x, *this)))
    {
        it = this->zeroValue();

        if (!this->hasZero())
        {
            ++m_size;
            this->setHasZero();
            this->zeroValue()->setHash(hash_value);
            inserted = true;
        }
        else
            inserted = false;

        return true;
    }

    return false;
adofsauron commented 1 year ago

During processing of row #i we will prefetch HashTable cell for row #(i + prefetch_look_ahead).

adofsauron commented 1 year ago 
        for (size_t i = row_begin; i < row_end; ++i)
            if (places[i])
                static_cast<const Derived *>(this)->add(places[i] + place_offset, columns, i, arena);
adofsauron commented 1 year ago

HUGE MMAP CRASH


#0  0x0000000002dcc26e in Tianmu::mm::TCMHeap::alloc (this=0x8b3ace0, size=1408) at /root/work/stonedb-dev-20230213/storage/tianmu/mm/tcm_heap_policy.cpp:76
#1  0x0000000002db8ccc in Tianmu::mm::MemoryHandling::alloc (this=0x825bf00, size=1408, type=Tianmu::mm::BLOCK_TYPE::BLOCK_HUGE, owner=0x7f4a7ca5b030, 
    nothrow=false) at /root/work/stonedb-dev-20230213/storage/tianmu/mm/memory_handling_policy.cpp:206
#2  0x0000000002dcd992 in Tianmu::mm::TraceableObject::alloc (this=0x7f4a7ca5b030, size=1408, type=Tianmu::mm::BLOCK_TYPE::BLOCK_HUGE, nothrow=false)
    at /root/work/stonedb-dev-20230213/storage/tianmu/mm/traceable_object.cpp:55
#3  0x000000000312db54 in Tianmu::core::MemBlockManager::GetBlock (this=0x7f4a7ca5b030)
    at /root/work/stonedb-dev-20230213/storage/tianmu/core/blocked_mem_table.cpp:45
#4  0x000000000312e3e1 in Tianmu::core::BlockedRowMemStorage::AddEmptyRow (this=0x7f4a7ca6ce98)
    at /root/work/stonedb-dev-20230213/storage/tianmu/core/blocked_mem_table.cpp:160
#5  0x00000000031162aa in Tianmu::core::ValueMatching_HashTable::FindCurrentRow (this=0x7f4a7ca6cdf0, input_buffer=0x7f4a7cb73a60 "", row=@0x7f52037ecbf8: 0, 
    add_if_new=true) at /root/work/stonedb-dev-20230213/storage/tianmu/core/value_matching_hashtable.cpp:202
#6  0x0000000002fd947a in Tianmu::core::GroupTable::FindCurrentRow (this=0x7f52497be2a8, row=@0x7f52037ecbf8: 0)
    at /root/work/stonedb-dev-20230213/storage/tianmu/core/group_table.cpp:412
#7  0x0000000002f84b34 in Tianmu::core::GroupByWrapper::FindCurrentRow (this=0x7f52497be1e0, row=@0x7f52037ecbf8: 0)
    at /root/work/stonedb-dev-20230213/storage/tianmu/core/groupby_wrapper.h:109
#8  0x0000000002f811f9 in Tianmu::core::AggregationAlgorithm::AggregatePackrow (this=0x7f52497be570, gbw=..., mit=0x7f52037ecd20, cur_tuple=0, 
    mem_used=0x7f52497bdc70) at /root/work/stonedb-dev-20230213/storage/tianmu/core/aggregation_algorithm.cpp:618
#9  0x0000000002f8238e in Tianmu::core::AggregationWorkerEnt::TaskAggrePacks (this=0x7f52497bdc80, taskIterator=0x7f4a7ca96b80, dims=0x7f52497bd800, 
    mit=0x7f52497bdcd0, task=0x7f4a7ca968f0, gbw=0x7f52497be1e0, ci=0x7f4a7ca39720, mem_used=0x7f52497bdc70)
    at /root/work/stonedb-dev-20230213/storage/tianmu/core/aggregation_algorithm.cpp:934
#10 0x0000000002f92efa in _ZSt13__invoke_implIvRMN6Tianmu4core20AggregationWorkerEntEFvPNS1_10MIIteratorEPNS1_15DimensionVectorES4_PNS1_5CTaskEPNS1_14GroupByWrapperEPNS1_11TransactionEPmERPS2_IRS4_RS6_SJ_RS8_RSA_RSC_RSD_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2_ (__f=
    @0x7f4a7ca68e48: (void (Tianmu::core::AggregationWorkerEnt::*)(Tianmu::core::AggregationWorkerEnt * const, Tianmu::core::MIIterator *, Tianmu::core::DimensionVector *, Tianmu::core::MIIterator *, Tianmu::core::CTask *, Tianmu::core::GroupByWrapper *, Tianmu::core::Transaction *, unsigned long *)) 0x2f8220e <Tianmu::core::AggregationWorkerEnt::TaskAggrePacks(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>, __t=@0x7f4a7ca68e90: 0x7f52497bdc80) at /usr/include/c++/8/bits/invoke.h:73
#11 0x0000000002f92a55 in std::__invoke<void (Tianmu::core::AggregationWorkerEnt::*&)(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*), Tianmu::core::AggregationWorkerEnt*&, Tianmu::core::MIIterator*&, Tianmu::core::DimensionVector*&, Tianmu::core::MIIterator*&, Tianmu::core::CTask*&, Tianmu::core::GroupByWrapper*&, Tianmu::core::Transaction*&, unsigned long*&> (__fn=
    @0x7f4a7ca68e48: (void (Tianmu::core::AggregationWorkerEnt::*)(Tianmu::core::AggregationWorkerEnt * const, Tianmu::core::MIIterator *, Tianmu::core::DimensionVector *, Tianmu::core::MIIterator *, Tianmu::core::CTask *, Tianmu::core::GroupByWrapper *, Tianmu::core::Transaction *, unsigned long *)) 0x2f8220e <Tianmu::core::AggregationWorkerEnt::TaskAggrePacks(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>) at /usr/include/c++/8/bits/invoke.h:95
#12 0x0000000002f9244e in std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>::__call<void, , 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul>) (this=0x7f4a7ca68e48, 
    __args=empty std::tuple) at /usr/include/c++/8/functional:400
#13 0x0000000002f9179a in std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>::operator()<, void>() (this=0x7f4a7ca68e48) at /usr/include/c++/8/functional:484
#14 0x0000000002f90f4d in std::__invoke_impl<void, std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIItera--Type <RET> for more, q to quit, c to continue without paging--
tor*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>&>(std::__invoke_other, std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>&) (__f=...) at /usr/include/c++/8/bits/invoke.h:60
#15 0x0000000002f90848 in std::__invoke<std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>&>(std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>&) (__fn=...)
    at /usr/include/c++/8/bits/invoke.h:95
#16 0x0000000002f901c7 in std::__future_base::_Task_state<std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>, std::allocator<int>, void ()>::_M_run()::{lambda()#1}::operator()() const (this=0x7f4a7ca68e20) at /usr/include/c++/8/future:1421
#17 0x0000000002f91841 in std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_state<std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>, std::allocator<int>, void ()>::_M_run()::{lambda()#1}, void>::operator()() const (this=0x7f52037ed590) at /usr/include/c++/8/future:1362

#18 0x0000000002f90fcc in std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_state<std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>, std::allocator<int>, void ()>::_M_run()::{lambda()#1}, void> >::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/8/bits/std_function.h:283
#19 0x0000000002c4f6bf in std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>::operator()() const (
    this=0x7f52037ed590) at /usr/include/c++/8/bits/std_function.h:687
#20 0x0000000002c4888b in std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) (this=0x7f4a7ca68e20, __f=0x7f52037ed590, __did_set=0x7f52037ed4f7) at /usr/include/c++/8/future:561
#21 0x0000000002c5eed1 in std::__invoke_impl<void, void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2<std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>*> (__f=
    @0x7f52037ed510: (void (std::__future_base::_State_baseV2::*)(std::__future_base::_State_baseV2 * const, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter>()> *, bool *)) 0x2c48864 <std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*)>, __t=@0x7f52037ed508: 0x7f4a7ca68e20) at /usr/include/c++/8/bits/invoke.h:73
#22 0x0000000002c5710f in std::__invoke<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__fn=
    @0x7f52037ed510: (void (std::__future_base::_State_baseV2::*)(std::__future_base::_State_baseV2 * const, std::function<std::unique_ptr<std::__future_base::_Resu--Type <RET> for more, q to quit, c to continue without paging--
lt_base, std::__future_base::_Result_base::_Deleter>()> *, bool *)) 0x2c48864 <std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*)>) at /usr/include/c++/8/bits/invoke.h:95
#23 0x0000000002c4f2bc in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#1}::operator()() const (this=0x7f52037ed480) at /usr/include/c++/8/mutex:672
#24 0x0000000002c4f327 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#2}::operator()() const (this=0x0) at /usr/include/c++/8/mutex:677
#25 0x0000000002c4f338 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#2}::_FUN() () at /usr/include/c++/8/mutex:677
#26 0x00007f5299101e67 in __pthread_once_slow () from /lib64/libpthread.so.0
#27 0x0000000002c42faf in __gthread_once (__once=0x7f4a7ca68e38, __func=0x7f52993d3b90 <__once_proxy>)
    at /usr/include/c++/8/x86_64-redhat-linux/bits/gthr-default.h:699
#28 0x0000000002c4f3e2 in std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__once=..., __f=
    @0x7f52037ed510: (void (std::__future_base::_State_baseV2::*)(std::__future_base::_State_baseV2 * const, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter>()> *, bool *)) 0x2c48864 <std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*)>) at /usr/include/c++/8/mutex:684
#29 0x0000000002c484d3 in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (this=0x7f4a7ca68e20, __res=..., __ignore_failure=false) at /usr/include/c++/8/future:401
#30 0x0000000002f90226 in std::__future_base::_Task_state<std::_Bind<void (Tianmu::core::AggregationWorkerEnt::*(Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*))(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*)>, std::allocator<int>, void ()>::_M_run() (this=0x7f4a7ca68e20) at /usr/include/c++/8/future:1423
#31 0x0000000002c5c48d in std::packaged_task<void ()>::operator()() (this=0x7f4a7ca98220) at /usr/include/c++/8/future:1556
#32 0x0000000002f8630a in Tianmu::utils::thread_pool::add_task<void (Tianmu::core::AggregationWorkerEnt::*)(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*), Tianmu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*&, Tianmu::core::Transaction*&, unsigned long*&>(void (Tianmu::core::AggregationWorkerEnt::*&&)(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*), Tianmu::core::AggregationWorkerEnt*&&, Tianmu::core::MIIterator*&&, Tianmu::core::DimensionVector*&&, Tianmu::core::MIIterator*&&, Tianmu::core::CTask*&&, Tianmu::core::GroupByWrapper*&, Tianmu::core::Transaction*&, unsigned long*&)::{lambda()#1}::operator()() const (this=0x7f4a7ca98260) at /root/work/stonedb-dev-20230213/storage/tianmu/util/thread_pool.h:94
#33 0x0000000002f8bf69 in std::_Function_handler<void (), Tianmu::utils::thread_pool::add_task<void (Tianmu::core::AggregationWorkerEnt::*)(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*), Tian--Type <RET> for more, q to quit, c to continue without paging--
mu::core::AggregationWorkerEnt*, Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*&, Tianmu::core::Transaction*&, unsigned long*&>(void (Tianmu::core::AggregationWorkerEnt::*&&)(Tianmu::core::MIIterator*, Tianmu::core::DimensionVector*, Tianmu::core::MIIterator*, Tianmu::core::CTask*, Tianmu::core::GroupByWrapper*, Tianmu::core::Transaction*, unsigned long*), Tianmu::core::AggregationWorkerEnt*&&, Tianmu::core::MIIterator*&&, Tianmu::core::DimensionVector*&&, Tianmu::core::MIIterator*&&, Tianmu::core::CTask*&&, Tianmu::core::GroupByWrapper*&, Tianmu::core::Transaction*&, unsigned long*&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/8/bits/std_function.h:297
#34 0x0000000002c4e1a4 in std::function<void ()>::operator()() const (this=0x7f52037ed6a0) at /usr/include/c++/8/bits/std_function.h:687
#35 0x0000000002c48d35 in Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}::operator()() const (__closure=0x827e7b8) at /root/work/stonedb-dev-20230213/storage/tianmu/util/thread_pool.h:61
#36 0x0000000002c6db9c in std::__invoke_impl<void, Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}>(std::__invoke_other, Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}&&) (__f=...) at /usr/include/c++/8/bits/invoke.h:60
#37 0x0000000002c66f47 in std::__invoke<Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}>(std::__invoke_result&&, (Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}&&)...) (__fn=...) at /usr/include/c++/8/bits/invoke.h:95
#38 0x0000000002c7c4a0 in std::thread::_Invoker<std::tuple<Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x827e7b8) at /usr/include/c++/8/thread:244
#39 0x0000000002c7bdf5 in std::thread::_Invoker<std::tuple<Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}> >::operator()() (this=0x827e7b8) at /usr/include/c++/8/thread:253
#40 0x0000000002c7af2c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<Tianmu::utils::thread_pool::thread_pool(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)::{lambda()#1}> > >::_M_run() (this=0x827e7b0) at /usr/include/c++/8/thread:196
#41 0x00007f52993d4b13 in execute_native_thread_routine () from /lib64/libstdc++.so.6
#42 0x00007f52990fa1ca in start_thread () from /lib64/libpthread.so.0
#43 0x00007f5296052e73 in clone () from /lib64/libc.so.6
adofsauron commented 1 year ago

6d48622ef3e31f9670062c4e7b7c45bf

adofsauron commented 1 year ago

linux death continues to occur when the disk is full using memory mapped files

f5ab86dc50ad9739cffb536b5d463f6a


Feb 27 10:42:44 kevin kernel: Out of memory: Killed process 4167 (mysqld) total-vm:57281236kB, anon-rss:15482648kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:100820kB oom_score_adj:0
adofsauron commented 1 year ago

When the disk space is sufficient, the system runs properly. Note that the physical memory is only 4GB. When the available memory is less than 1GB, memory mapping to disk files occurs

25F0A771-CCD6-4317-BF4A-C0E5150B9697

adofsauron commented 1 year ago

7fc71f1ce000-7fce9f1ce000 rw-s 00000000 08:02 3670018 /tmp/tianmuhuge.12039