Closed badboynt1 closed 1 month ago
跑一轮tpch1T, fileservice相关的堆外分配大约有5.5T。 明显不应该这么多,因为现在scan算子可以复用自己的内存。 (pprof) top 50 --cum Showing nodes accounting for 6142.95GB, 99.75% of 6158.53GB total Dropped 159 nodes (cum <= 30.79GB) flat flat% sum% cum cum% 5542.88GB 90.00% 90.00% 5542.88GB 90.00% | stack omitted | 0 0% 90.00% 615.65GB 10.00% runtime.goexit 0 0% 90.00% 615.41GB 9.99% github.com/panjf2000/ants/v2.(*goWorker).run.func1 0 0% 90.00% 615.37GB 9.99% github.com/matrixorigin/matrixone/pkg/sql/compile.(*Scope).MergeRun.func1 0 0% 90.00% 615GB 9.99% github.com/matrixorigin/matrixone/pkg/sql/compile.(*Scope).Run 0 0% 90.00% 615GB 9.99% github.com/matrixorigin/matrixone/pkg/vm/pipeline.(*Pipeline).run 0 0% 90.00% 605.09GB 9.83% github.com/matrixorigin/matrixone/pkg/vm.ChildrenCall 0 0% 90.00% 605.05GB 9.82% github.com/matrixorigin/matrixone/pkg/sql/colexec/projection.(*Projection).Call 600.08GB 9.74% 99.75% 600.08GB 9.74% github.com/matrixorigin/matrixone/pkg/common/malloc.(*MetricsAllocator[...]).Allocate 0 0% 99.75% 600.08GB 9.74% github.com/matrixorigin/matrixone/pkg/common/malloc.(*RandomAllocator[...]).Allocate 0 0% 99.75% 600.08GB 9.74% github.com/matrixorigin/matrixone/pkg/fileservice.(*LocalFS).Read 0 0% 99.75% 600.08GB 9.74% github.com/matrixorigin/matrixone/pkg/fileservice.(*LocalFS).read 0 0% 99.75% 599.47GB 9.73% github.com/matrixorigin/matrixone/pkg/objectio.ReadOneBlockWithMeta 0 0% 99.75% 599.47GB 9.73% github.com/matrixorigin/matrixone/pkg/objectio.ReadOneBlock 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/pipeline.(*Pipeline).Run 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/sql/colexec/table_scan.(*TableScan).Call 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/disttae.(*reader).Read 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.BlockDataRead 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.LoadColumnsData 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.LoadColumns 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.readBlockData 0 0% 99.75% 599.43GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.readBlockData.func1 0 0% 99.75% 599.33GB 9.73% github.com/matrixorigin/matrixone/pkg/vm/engine/tae/blockio.BlockDataReadInner
No response
不是问题, close
Is there an existing issue for the same tech request?
Does this tech request not affect user experience?
What would you like to be added ?
Why is this needed ?
No response
Additional information
No response