Closed czentgr closed 6 months ago
FYI @majetideepak
A different stack trace I see:
*** Aborted at 1710860656 (unix time) try "date -d @1710860656" if you are using GNU date *** |
-- | --
| PC: @ 0x0 (unknown) |
| *** SIGSEGV (@0x18) received by PID 8 (TID 0x7d9a037fe700) from PID 24; stack trace: *** |
| @ 0x7f883794dd20 (unknown) |
| @ 0x7f8835cbcae7 __memmove_evex_unaligned_erms |
| @ 0x39f6a5c facebook::velox::exec::RowContainer::initializeRow() |
| @ 0x3b39219 _ZZN8facebook5velox4exec9HashBuild8addInputESt10shared_ptrINS0_9RowVectorEEENKUlT_E_clIiEEDaS6_ |
| @ 0x3b42791 facebook::velox::exec::HashBuild::addInput() |
| @ 0x39cbd44 facebook::velox::exec::Driver::runInternal() |
| @ 0x39ce8cd facebook::velox::exec::Driver::run() |
| @ 0x39cea8f _ZN5folly6detail8function14FunctionTraitsIFvvEE9callSmallIZN8facebook5velox4exec6Driver7enqueueESt10shared_ptrIS9_EEUlvE_EEvRNS1_4DataE |
| @ 0x4141b67 folly::detail::function::FunctionTraits<>::operator()() |
| @ 0x7f8839d68024 folly::ThreadPoolExecutor::runTask() |
| @ 0x413dc53 folly::CPUThreadPoolExecutor::threadRun() |
| @ 0x7f8839d7486b std::__invoke_impl<>() |
| @ 0x7f8839d73239 std::__invoke<>() |
| @ 0x7f8839d7160b _ZNSt5_BindIFMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEPS1_S4_EE6__callIvJEJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE |
| W20240319 15:04:16.895465 4159 MmapAllocator.cpp:79] [MEM] Exceeded memory allocator limit when allocating 32 pages with capacity of 57409536 pages |
| @ 0x7f8839d6fc40 std::_Bind<>::operator()<>() |
| @ 0x7f8839d6e210 folly::detail::function::FunctionTraits<>::callSmall<>() |
| @ 0x7f8836636b23 (unknown) |
| @ 0x7f88379431ca start_thread |
| @ 0x7f8835c3d8d3 __GI___clone |
| @ 0x0 (unknown)
Yes, I think ultimately it comes from the same source.
The stack to lead up to it from RowContainer::newRow
which calls the initializeRow
function that crashes.
MmapAllocator::allocateContiguousImpl
MmapAllocator::allocateContiguousWithoutRetry
MemoryAllocator::allocateContiguous
MemoryPoolImpl::allocateContiguous
AllocationPool::newRunImpl
AllocationPool::allocateFixed
RowContainer::newRow
The MmapAllocator::allocateContiguousImpl
returns an Allocation
that points to -1
. Later the pointer is used first in initializeRow
when the memset
is called.
After we handle the MAP_FAILED
case and return false in that function, the SEGV issue is gone. Meanwhile we output the memory stats when data is MAP_FAILED
.
The benchmark run was tpcds10k and on 16nodes cluster with 219 system-memory-gb setting for each node.
W20240323 04:37:37.417330 3268 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 8 pages with capacity of 57409536 pages
W20240323 04:37:43.431779 3337 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 16 pages with capacity of 57409536 pages
I20240323 04:37:44.118325 19 PeriodicServiceInventoryManager.cpp:115] Announcement succeeded: HTTP 202. State: active.
W20240323 04:37:44.954460 3326 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 128 pages with capacity of 57409536 pages
W20240323 04:37:47.917161 3377 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 256 pages with capacity of 57409536 pages
W20240323 04:37:55.762253 3528 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 16 pages with capacity of 57409536 pages
W20240323 04:38:01.240581 3359 MmapAllocator.cpp:80] [MEM] Exceeded memory allocator limit when allocating 128 pages with capacity of 57409536 pages
I20240323 04:38:08.545166 39 PeriodicTaskManager.cpp:636] Spill memory usage: current[0B] peak[0B]
I20240323 04:38:08.580164 40 PeriodicTaskManager.cpp:593] Updated memory arbitrator stats: STATS[numRequests 6176 numSucceeded 6176 numAborted 0 numFailures 0 numNonReclaimableAttempts 0 numReserves 73 numReleases 72 queueTime 3m 5s arbitrationTime 3m 28s reclaimTime 0us shrunkMemory 0B reclaimedMemory 0B maxCapacity 208.00GB freeCapacity 196.61GB]
I20240323 04:38:08.580229 40 PeriodicTaskManager.cpp:595] Memory arbitrator stats change: STATS[numRequests 45 numSucceeded 45 numAborted 0 numFailures 0 numNonReclaimableAttempts 0 numReserves 0 numReleases 0 queueTime 52.56ms arbitrationTime 80.72ms reclaimTime 0us shrunkMemory 0B reclaimedMemory 0B maxCapacity 208.00GB freeCapacity 196.61GB]
I20240323 04:38:08.796352 35 PeriodicTaskManager.cpp:413] Cache stats:
Cache size: 207.92GB tinySize: 748.92KB large size: 207.92GB
Cache entries: 51106 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2409
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75951 eviction checks: 445371 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
E20240323 04:38:08.921046 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 125.41MB allocated pages 57377430 mapped pages 57409432 external mapped pages 2765334
[size 1: 20876(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18779(293MB) allocated 18827 mapped]
[size 8: 21199(662MB) allocated 21281 mapped]
[size 16: 7984(499MB) allocated 8311 mapped]
[size 32: 18627(2328MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23836(11918MB) allocated 23869 mapped]
[size 256: 190706(190706MB) allocated 190715 mapped]
]
E20240323 04:38:08.925364 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.92GB tinySize: 748.92KB large size: 207.92GB
Cache entries: 51106 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2409
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75951 eviction checks: 445371 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57378541 cached pages: 54504065
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 121.07MB allocated pages 57378541 mapped pages 57409432 external mapped pages 2765334
[size 1: 20875(81MB) allocated 21540 mapped]
[size 2: 16872(131MB) allocated 22397 mapped]
[size 4: 18783(293MB) allocated 18827 mapped]
[size 8: 21201(662MB) allocated 21281 mapped]
[size 16: 7961(497MB) allocated 8311 mapped]
[size 32: 18633(2329MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23836(11918MB) allocated 23869 mapped]
[size 256: 190706(190706MB) allocated 190715 mapped]
]
E20240323 04:38:08.931684 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.931730 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 123.25MB allocated pages 57377984 mapped pages 57409432 external mapped pages 2765334
[size 1: 20874(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18780(293MB) allocated 18827 mapped]
[size 8: 21204(662MB) allocated 21281 mapped]
[size 16: 7960(497MB) allocated 8311 mapped]
[size 32: 18633(2329MB) allocated 18690 mapped]
[size 64: 26855(6713MB) allocated 26920 mapped]
[size 128: 23836(11918MB) allocated 23869 mapped]
[size 256: 190703(190703MB) allocated 190715 mapped]
]
E20240323 04:38:08.934777 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.91GB tinySize: 748.89KB large size: 207.91GB
Cache entries: 51104 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2411
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75953 eviction checks: 445373 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57377958 cached pages: 54503150
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 123.35MB allocated pages 57377958 mapped pages 57409432 external mapped pages 2765334
[size 1: 20874(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18781(293MB) allocated 18827 mapped]
[size 8: 21204(662MB) allocated 21281 mapped]
[size 16: 7958(497MB) allocated 8311 mapped]
[size 32: 18633(2329MB) allocated 18690 mapped]
[size 64: 26855(6713MB) allocated 26920 mapped]
[size 128: 23836(11918MB) allocated 23869 mapped]
[size 256: 190703(190703MB) allocated 190715 mapped]
]
E20240323 04:38:08.940786 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.940829 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 135.11MB allocated pages 57374947 mapped pages 57409432 external mapped pages 2765334
[size 1: 20873(81MB) allocated 21540 mapped]
[size 2: 16872(131MB) allocated 22397 mapped]
[size 4: 18781(293MB) allocated 18827 mapped]
[size 8: 21204(662MB) allocated 21281 mapped]
[size 16: 7963(497MB) allocated 8311 mapped]
[size 32: 18632(2329MB) allocated 18690 mapped]
[size 64: 26854(6713MB) allocated 26920 mapped]
[size 128: 23833(11916MB) allocated 23869 mapped]
[size 256: 190693(190693MB) allocated 190715 mapped]
]
E20240323 04:38:08.943756 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.90GB tinySize: 748.83KB large size: 207.90GB
Cache entries: 51100 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2415
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75957 eviction checks: 445377 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57375750 cached pages: 54500107
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 131.98MB allocated pages 57375750 mapped pages 57409432 external mapped pages 2765334
[size 1: 20874(81MB) allocated 21540 mapped]
[size 2: 16876(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21211(662MB) allocated 21281 mapped]
[size 16: 7975(498MB) allocated 8311 mapped]
[size 32: 18636(2329MB) allocated 18690 mapped]
[size 64: 26860(6715MB) allocated 26920 mapped]
[size 128: 23833(11916MB) allocated 23869 mapped]
[size 256: 190693(190693MB) allocated 190715 mapped]
]
E20240323 04:38:08.949721 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.949761 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 148.25MB allocated pages 57371583 mapped pages 57409432 external mapped pages 2765334
[size 1: 20873(81MB) allocated 21540 mapped]
[size 2: 16876(131MB) allocated 22397 mapped]
[size 4: 18783(293MB) allocated 18827 mapped]
[size 8: 21211(662MB) allocated 21281 mapped]
[size 16: 7963(497MB) allocated 8311 mapped]
[size 32: 18636(2329MB) allocated 18690 mapped]
[size 64: 26858(6714MB) allocated 26920 mapped]
[size 128: 23831(11915MB) allocated 23869 mapped]
[size 256: 190679(190679MB) allocated 190715 mapped]
]
E20240323 04:38:08.952646 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.89GB tinySize: 748.79KB large size: 207.89GB
Cache entries: 51097 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2418
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75960 eviction checks: 445385 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57371703 cached pages: 54496130
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 147.79MB allocated pages 57371703 mapped pages 57409432 external mapped pages 2765334
[size 1: 20873(81MB) allocated 21540 mapped]
[size 2: 16876(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21211(662MB) allocated 21281 mapped]
[size 16: 7971(498MB) allocated 8311 mapped]
[size 32: 18636(2329MB) allocated 18690 mapped]
[size 64: 26858(6714MB) allocated 26920 mapped]
[size 128: 23831(11915MB) allocated 23869 mapped]
[size 256: 190679(190679MB) allocated 190715 mapped]
]
E20240323 04:38:08.958513 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.958555 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 160.50MB allocated pages 57368449 mapped pages 57409432 external mapped pages 2765334
[size 1: 20873(81MB) allocated 21540 mapped]
[size 2: 16875(131MB) allocated 22397 mapped]
[size 4: 18781(293MB) allocated 18827 mapped]
[size 8: 21209(662MB) allocated 21281 mapped]
[size 16: 7976(498MB) allocated 8311 mapped]
[size 32: 18635(2329MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23830(11915MB) allocated 23869 mapped]
[size 256: 190667(190667MB) allocated 190715 mapped]
]
E20240323 04:38:08.961386 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.87GB tinySize: 748.76KB large size: 207.87GB
Cache entries: 51095 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2420
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75962 eviction checks: 445390 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57368641 cached pages: 54492812
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 159.75MB allocated pages 57368641 mapped pages 57409432 external mapped pages 2765334
[size 1: 20873(81MB) allocated 21540 mapped]
[size 2: 16875(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 7989(499MB) allocated 8311 mapped]
[size 32: 18643(2330MB) allocated 18690 mapped]
[size 64: 26858(6714MB) allocated 26920 mapped]
[size 128: 23830(11915MB) allocated 23869 mapped]
[size 256: 190667(190667MB) allocated 190715 mapped]
]
E20240323 04:38:08.967202 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.967252 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 169.36MB allocated pages 57366179 mapped pages 57409432 external mapped pages 2765334
[size 1: 20871(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21212(662MB) allocated 21281 mapped]
[size 16: 7982(498MB) allocated 8311 mapped]
[size 32: 18640(2330MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23830(11915MB) allocated 23869 mapped]
[size 256: 190657(190657MB) allocated 190715 mapped]
]
E20240323 04:38:08.970023 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.86GB tinySize: 748.64KB large size: 207.86GB
Cache entries: 51087 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2428
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75970 eviction checks: 445398 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57366311 cached pages: 54490057
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 168.85MB allocated pages 57366311 mapped pages 57409432 external mapped pages 2765334
[size 1: 20871(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21212(662MB) allocated 21281 mapped]
[size 16: 7991(499MB) allocated 8311 mapped]
[size 32: 18640(2330MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23830(11915MB) allocated 23869 mapped]
[size 256: 190657(190657MB) allocated 190715 mapped]
]
E20240323 04:38:08.975678 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.975719 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 179.29MB allocated pages 57363639 mapped pages 57409432 external mapped pages 2765334
[size 1: 20871(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18782(293MB) allocated 18827 mapped]
[size 8: 21212(662MB) allocated 21281 mapped]
[size 16: 7997(499MB) allocated 8311 mapped]
[size 32: 18640(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190647(190647MB) allocated 190715 mapped]
]
E20240323 04:38:08.978440 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.85GB tinySize: 748.60KB large size: 207.85GB
Cache entries: 51084 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2431
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75973 eviction checks: 445401 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57363607 cached pages: 54487289
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 179.41MB allocated pages 57363607 mapped pages 57409432 external mapped pages 2765334
[size 1: 20871(81MB) allocated 21540 mapped]
[size 2: 16873(131MB) allocated 22397 mapped]
[size 4: 18784(293MB) allocated 18827 mapped]
[size 8: 21214(662MB) allocated 21281 mapped]
[size 16: 8013(500MB) allocated 8311 mapped]
[size 32: 18643(2330MB) allocated 18690 mapped]
[size 64: 26864(6716MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190647(190647MB) allocated 190715 mapped]
]
E20240323 04:38:08.984169 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.984211 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 187.32MB allocated pages 57361581 mapped pages 57409432 external mapped pages 2765334
[size 1: 20871(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21214(662MB) allocated 21281 mapped]
[size 16: 8018(501MB) allocated 8311 mapped]
[size 32: 18647(2330MB) allocated 18690 mapped]
[size 64: 26860(6715MB) allocated 26920 mapped]
[size 128: 23831(11915MB) allocated 23869 mapped]
[size 256: 190638(190638MB) allocated 190715 mapped]
]
E20240323 04:38:08.986992 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.84GB tinySize: 748.54KB large size: 207.84GB
Cache entries: 51080 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2435
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75977 eviction checks: 445405 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57361885 cached pages: 54484431
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 186.14MB allocated pages 57361885 mapped pages 57409432 external mapped pages 2765334
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18787(293MB) allocated 18827 mapped]
[size 8: 21214(662MB) allocated 21281 mapped]
[size 16: 7984(499MB) allocated 8311 mapped]
[size 32: 18647(2330MB) allocated 18690 mapped]
[size 64: 26860(6715MB) allocated 26920 mapped]
[size 128: 23831(11915MB) allocated 23869 mapped]
[size 256: 190638(190638MB) allocated 190715 mapped]
]
E20240323 04:38:08.992779 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.992820 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 202.25MB allocated pages 57357761 mapped pages 57409432 external mapped pages 2765334
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 7986(499MB) allocated 8311 mapped]
[size 32: 18646(2330MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23828(11914MB) allocated 23869 mapped]
[size 256: 190624(190624MB) allocated 190713 mapped]
]
E20240323 04:38:08.993736 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 200.12MB allocated pages 57358305 mapped pages 57409432 external mapped pages 2765846
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 7989(499MB) allocated 8311 mapped]
[size 32: 18646(2330MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23828(11914MB) allocated 23869 mapped]
[size 256: 190624(190624MB) allocated 190713 mapped]
]
E20240323 04:38:08.995599 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.83GB tinySize: 748.48KB large size: 207.83GB
Cache entries: 51076 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2439
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75981 eviction checks: 445409 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57358449 cached pages: 54480195
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 199.56MB allocated pages 57358449 mapped pages 57409432 external mapped pages 2765846
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 7998(499MB) allocated 8311 mapped]
[size 32: 18646(2330MB) allocated 18690 mapped]
[size 64: 26857(6714MB) allocated 26920 mapped]
[size 128: 23828(11914MB) allocated 23869 mapped]
[size 256: 190624(190624MB) allocated 190713 mapped]
]
E20240323 04:38:09.001332 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:08.996912 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.83GB tinySize: 748.48KB large size: 207.83GB
Cache entries: 51076 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2439
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75981 eviction checks: 445409 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096273
Allocated pages: 57358497 cached pages: 54480195
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 199.37MB allocated pages 57358497 mapped pages 57409432 external mapped pages 2765846
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 8000(500MB) allocated 8311 mapped]
[size 32: 18644(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23828(11914MB) allocated 23869 mapped]
[size 256: 190614(190614MB) allocated 190713 mapped]
]
E20240323 04:38:09.002688 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.002051 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 209.70MB allocated pages 57355854 mapped pages 57409432 external mapped pages 2765846
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16869(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 8004(500MB) allocated 8311 mapped]
[size 32: 18644(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190607(190607MB) allocated 190713 mapped]
]
E20240323 04:38:09.002728 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 217.07MB allocated pages 57353966 mapped pages 57409432 external mapped pages 2765846
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16869(131MB) allocated 22397 mapped]
[size 4: 18786(293MB) allocated 18827 mapped]
[size 8: 21213(662MB) allocated 21281 mapped]
[size 16: 8007(500MB) allocated 8311 mapped]
[size 32: 18644(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190607(190607MB) allocated 190713 mapped]
]
E20240323 04:38:09.004604 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.81GB tinySize: 748.34KB large size: 207.81GB
Cache entries: 51066 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2449
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75991 eviction checks: 445419 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57354141 cached pages: 54475568
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 216.39MB allocated pages 57354141 mapped pages 57409432 external mapped pages 2765846
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18787(293MB) allocated 18827 mapped]
[size 8: 21214(662MB) allocated 21281 mapped]
[size 16: 8019(501MB) allocated 8311 mapped]
[size 32: 18644(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190607(190607MB) allocated 190713 mapped]
]
E20240323 04:38:09.010218 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
I20240323 04:38:09.010246 3751 AsyncDataCache.cpp:799] [CACHE] Backoff in allocation contention for 23.67ms
E20240323 04:38:09.005873 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.81GB tinySize: 748.34KB large size: 207.81GB
Cache entries: 51066 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2449
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75991 eviction checks: 445419 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57354205 cached pages: 54475568
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 216.14MB allocated pages 57354205 mapped pages 57409432 external mapped pages 2765846
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16870(131MB) allocated 22397 mapped]
[size 4: 18791(293MB) allocated 18827 mapped]
[size 8: 21217(663MB) allocated 21281 mapped]
[size 16: 8028(501MB) allocated 8311 mapped]
[size 32: 18645(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190607(190607MB) allocated 190713 mapped]
]
E20240323 04:38:09.011552 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.011590 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 225.28MB allocated pages 57351864 mapped pages 57408920 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16872(131MB) allocated 22397 mapped]
[size 4: 18790(293MB) allocated 18827 mapped]
[size 8: 21216(663MB) allocated 21281 mapped]
[size 16: 8030(501MB) allocated 8311 mapped]
[size 32: 18645(2330MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190599(190599MB) allocated 190713 mapped]
]
E20240323 04:38:09.014307 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.80GB tinySize: 748.32KB large size: 207.80GB
Cache entries: 51065 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2450
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75992 eviction checks: 445423 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57352203 cached pages: 54473520
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 223.96MB allocated pages 57352203 mapped pages 57408920 external mapped pages 2765334
[size 1: 20879(81MB) allocated 21540 mapped]
[size 2: 16875(131MB) allocated 22397 mapped]
[size 4: 18793(293MB) allocated 18827 mapped]
[size 8: 21222(663MB) allocated 21281 mapped]
[size 16: 8058(503MB) allocated 8311 mapped]
[size 32: 18653(2331MB) allocated 18690 mapped]
[size 64: 26856(6714MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190599(190599MB) allocated 190713 mapped]
]
E20240323 04:38:09.020015 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.020058 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 234.78MB allocated pages 57349432 mapped pages 57408920 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16874(131MB) allocated 22397 mapped]
[size 4: 18798(293MB) allocated 18827 mapped]
[size 8: 21224(663MB) allocated 21281 mapped]
[size 16: 8070(504MB) allocated 8311 mapped]
[size 32: 18654(2331MB) allocated 18690 mapped]
[size 64: 26855(6713MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190588(190588MB) allocated 190713 mapped]
]
E20240323 04:38:09.022784 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.79GB tinySize: 748.29KB large size: 207.79GB
Cache entries: 51063 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2452
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 75994 eviction checks: 445432 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57351018 cached pages: 54470377
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 228.59MB allocated pages 57351018 mapped pages 57408920 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16877(131MB) allocated 22397 mapped]
[size 4: 18804(293MB) allocated 18827 mapped]
[size 8: 21231(663MB) allocated 21281 mapped]
[size 16: 8089(505MB) allocated 8311 mapped]
[size 32: 18665(2333MB) allocated 18690 mapped]
[size 64: 26862(6715MB) allocated 26920 mapped]
[size 128: 23831(11915MB) allocated 23869 mapped]
[size 256: 190588(190588MB) allocated 190713 mapped]
]
E20240323 04:38:09.028748 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.028807 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 235.23MB allocated pages 57349317 mapped pages 57408920 external mapped pages 2765334
[size 1: 20885(81MB) allocated 21540 mapped]
[size 2: 16878(131MB) allocated 22397 mapped]
[size 4: 18808(293MB) allocated 18827 mapped]
[size 8: 21233(663MB) allocated 21281 mapped]
[size 16: 8081(505MB) allocated 8311 mapped]
[size 32: 18660(2332MB) allocated 18690 mapped]
[size 64: 26864(6716MB) allocated 26920 mapped]
[size 128: 23827(11913MB) allocated 23869 mapped]
[size 256: 190585(190585MB) allocated 190713 mapped]
]
E20240323 04:38:09.034021 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 242.97MB allocated pages 57347336 mapped pages 57409432 external mapped pages 2765846
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16886(131MB) allocated 22397 mapped]
[size 4: 18812(293MB) allocated 18827 mapped]
[size 8: 21237(663MB) allocated 21281 mapped]
[size 16: 8107(506MB) allocated 8311 mapped]
[size 32: 18663(2332MB) allocated 18690 mapped]
[size 64: 26867(6716MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190570(190570MB) allocated 190713 mapped]
]
E20240323 04:38:09.031642 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.76GB tinySize: 748.16KB large size: 207.76GB
Cache entries: 51054 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2461
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76003 eviction checks: 445441 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57347432 cached pages: 54463910
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 242.59MB allocated pages 57347432 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16885(131MB) allocated 22397 mapped]
[size 4: 18811(293MB) allocated 18827 mapped]
[size 8: 21238(663MB) allocated 21281 mapped]
[size 16: 8036(502MB) allocated 8311 mapped]
[size 32: 18663(2332MB) allocated 18690 mapped]
[size 64: 26867(6716MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190570(190570MB) allocated 190713 mapped]
]
E20240323 04:38:09.038813 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.039031 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 261.77MB allocated pages 57342523 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16888(131MB) allocated 22397 mapped]
[size 4: 18812(293MB) allocated 18827 mapped]
[size 8: 21241(663MB) allocated 21281 mapped]
[size 16: 8042(502MB) allocated 8311 mapped]
[size 32: 18667(2333MB) allocated 18690 mapped]
[size 64: 26873(6718MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190554(190554MB) allocated 190713 mapped]
]
E20240323 04:38:09.037946 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.76GB tinySize: 748.16KB large size: 207.76GB
Cache entries: 51054 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2461
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76003 eviction checks: 445441 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57342991 cached pages: 54459814
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 259.94MB allocated pages 57342991 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16885(131MB) allocated 22397 mapped]
[size 4: 18817(294MB) allocated 18827 mapped]
[size 8: 21241(663MB) allocated 21281 mapped]
[size 16: 8036(502MB) allocated 8311 mapped]
[size 32: 18670(2333MB) allocated 18690 mapped]
[size 64: 26874(6718MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190554(190554MB) allocated 190713 mapped]
]
E20240323 04:38:09.044456 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.041893 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.75GB tinySize: 748.13KB large size: 207.75GB
Cache entries: 51052 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2463
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76005 eviction checks: 445443 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57342335 cached pages: 54459814
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 262.50MB allocated pages 57342335 mapped pages 57408920 external mapped pages 2765334
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16886(131MB) allocated 22397 mapped]
[size 4: 18818(294MB) allocated 18827 mapped]
[size 8: 21240(663MB) allocated 21281 mapped]
[size 16: 8031(501MB) allocated 8311 mapped]
[size 32: 18670(2333MB) allocated 18690 mapped]
[size 64: 26874(6718MB) allocated 26920 mapped]
[size 128: 23825(11912MB) allocated 23869 mapped]
[size 256: 190554(190554MB) allocated 190713 mapped]
]
E20240323 04:38:09.047709 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.047753 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 272.65MB allocated pages 57339738 mapped pages 57408920 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16889(131MB) allocated 22397 mapped]
[size 4: 18818(294MB) allocated 18827 mapped]
[size 8: 21247(663MB) allocated 21281 mapped]
[size 16: 8042(502MB) allocated 8311 mapped]
[size 32: 18677(2334MB) allocated 18690 mapped]
[size 64: 26870(6717MB) allocated 26920 mapped]
[size 128: 23822(11911MB) allocated 23869 mapped]
[size 256: 190546(190546MB) allocated 190713 mapped]
]
E20240323 04:38:09.051606 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 279.14MB allocated pages 57338075 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16888(131MB) allocated 22397 mapped]
[size 4: 18822(294MB) allocated 18827 mapped]
[size 8: 21250(664MB) allocated 21281 mapped]
[size 16: 8027(501MB) allocated 8311 mapped]
[size 32: 18681(2335MB) allocated 18690 mapped]
[size 64: 26871(6717MB) allocated 26920 mapped]
[size 128: 23822(11911MB) allocated 23869 mapped]
[size 256: 190536(190536MB) allocated 190713 mapped]
]
E20240323 04:38:09.050536 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.73GB tinySize: 747.99KB large size: 207.73GB
Cache entries: 51042 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2473
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76015 eviction checks: 445453 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57338139 cached pages: 54454282
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 278.89MB allocated pages 57338139 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16888(131MB) allocated 22397 mapped]
[size 4: 18822(294MB) allocated 18827 mapped]
[size 8: 21250(664MB) allocated 21281 mapped]
[size 16: 8029(501MB) allocated 8311 mapped]
[size 32: 18681(2335MB) allocated 18690 mapped]
[size 64: 26871(6717MB) allocated 26920 mapped]
[size 128: 23822(11911MB) allocated 23869 mapped]
[size 256: 190536(190536MB) allocated 190713 mapped]
]
E20240323 04:38:09.056095 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.056138 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 294.46MB allocated pages 57334154 mapped pages 57409432 external mapped pages 2765846
[size 1: 20876(81MB) allocated 21540 mapped]
[size 2: 16886(131MB) allocated 22397 mapped]
[size 4: 18821(294MB) allocated 18827 mapped]
[size 8: 21249(664MB) allocated 21281 mapped]
[size 16: 8035(502MB) allocated 8311 mapped]
[size 32: 18678(2334MB) allocated 18690 mapped]
[size 64: 26869(6717MB) allocated 26920 mapped]
[size 128: 23820(11910MB) allocated 23869 mapped]
[size 256: 190522(190522MB) allocated 190713 mapped]
]
E20240323 04:38:09.055198 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.71GB tinySize: 747.93KB large size: 207.71GB
Cache entries: 51038 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2477
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76019 eviction checks: 445457 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57334313 cached pages: 54450201
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 293.84MB allocated pages 57334313 mapped pages 57409432 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16888(131MB) allocated 22397 mapped]
[size 4: 18826(294MB) allocated 18827 mapped]
[size 8: 21253(664MB) allocated 21281 mapped]
[size 16: 8052(503MB) allocated 8311 mapped]
[size 32: 18679(2334MB) allocated 18690 mapped]
[size 64: 26869(6717MB) allocated 26920 mapped]
[size 128: 23820(11910MB) allocated 23869 mapped]
[size 256: 190522(190522MB) allocated 190713 mapped]
]
E20240323 04:38:09.063012 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.058771 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.71GB tinySize: 747.93KB large size: 207.71GB
Cache entries: 51038 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2477
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76019 eviction checks: 445457 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57334562 cached pages: 54450201
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 292.87MB allocated pages 57334562 mapped pages 57409432 external mapped pages 2765846
[size 1: 20876(81MB) allocated 21540 mapped]
[size 2: 16887(131MB) allocated 22397 mapped]
[size 4: 18825(294MB) allocated 18827 mapped]
[size 8: 21252(664MB) allocated 21281 mapped]
[size 16: 8050(503MB) allocated 8311 mapped]
[size 32: 18679(2334MB) allocated 18690 mapped]
[size 64: 26869(6717MB) allocated 26920 mapped]
[size 128: 23820(11910MB) allocated 23869 mapped]
[size 256: 190522(190522MB) allocated 190713 mapped]
]
E20240323 04:38:09.064047 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.064111 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 310.82MB allocated pages 57329965 mapped pages 57408920 external mapped pages 2765334
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16889(131MB) allocated 22397 mapped]
[size 4: 18830(294MB) allocated 18830 mapped]
[size 8: 21255(664MB) allocated 21281 mapped]
[size 16: 8064(504MB) allocated 8311 mapped]
[size 32: 18680(2335MB) allocated 18690 mapped]
[size 64: 26869(6717MB) allocated 26920 mapped]
[size 128: 23820(11910MB) allocated 23869 mapped]
[size 256: 190506(190506MB) allocated 190713 mapped]
]
E20240323 04:38:09.066720 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.70GB tinySize: 747.88KB large size: 207.70GB
Cache entries: 51035 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2480
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76022 eviction checks: 445460 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57330379 cached pages: 54446095
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 309.21MB allocated pages 57330379 mapped pages 57408932 external mapped pages 2765334
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16892(131MB) allocated 22397 mapped]
[size 4: 18830(294MB) allocated 18830 mapped]
[size 8: 21255(664MB) allocated 21281 mapped]
[size 16: 8085(505MB) allocated 8311 mapped]
[size 32: 18680(2335MB) allocated 18690 mapped]
[size 64: 26869(6717MB) allocated 26920 mapped]
[size 128: 23820(11910MB) allocated 23869 mapped]
[size 256: 190506(190506MB) allocated 190713 mapped]
]
E20240323 04:38:09.072160 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.072201 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 323.14MB allocated pages 57326811 mapped pages 57408932 external mapped pages 2765334
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16896(132MB) allocated 22397 mapped]
[size 4: 18835(294MB) allocated 18838 mapped]
[size 8: 21269(664MB) allocated 21281 mapped]
[size 16: 8097(506MB) allocated 8311 mapped]
[size 32: 18689(2336MB) allocated 18697 mapped]
[size 64: 26877(6719MB) allocated 26920 mapped]
[size 128: 23815(11907MB) allocated 23869 mapped]
[size 256: 190495(190495MB) allocated 190713 mapped]
]
E20240323 04:38:09.075006 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.68GB tinySize: 747.81KB large size: 207.68GB
Cache entries: 51030 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2485
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76027 eviction checks: 445465 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57328392 cached pages: 54442319
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 316.97MB allocated pages 57328392 mapped pages 57409196 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16902(132MB) allocated 22397 mapped]
[size 4: 18840(294MB) allocated 18840 mapped]
[size 8: 21280(665MB) allocated 21281 mapped]
[size 16: 8069(504MB) allocated 8311 mapped]
[size 32: 18695(2336MB) allocated 18697 mapped]
[size 64: 26872(6718MB) allocated 26920 mapped]
[size 128: 23821(11910MB) allocated 23869 mapped]
[size 256: 190495(190495MB) allocated 190713 mapped]
]
E20240323 04:38:09.080940 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.096529 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 328.47MB allocated pages 57325448 mapped pages 57409412 external mapped pages 2765334
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16901(132MB) allocated 22397 mapped]
[size 4: 18830(294MB) allocated 18842 mapped]
[size 8: 21251(664MB) allocated 21287 mapped]
[size 16: 8067(504MB) allocated 8311 mapped]
[size 32: 18664(2333MB) allocated 18702 mapped]
[size 64: 26853(6713MB) allocated 26920 mapped]
[size 128: 23818(11909MB) allocated 23869 mapped]
[size 256: 190477(190477MB) allocated 190711 mapped]
]
E20240323 04:38:09.097841 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 344.23MB allocated pages 57321412 mapped pages 57409412 external mapped pages 2765846
[size 1: 20877(81MB) allocated 21540 mapped]
[size 2: 16900(132MB) allocated 22397 mapped]
[size 4: 18829(294MB) allocated 18842 mapped]
[size 8: 21250(664MB) allocated 21287 mapped]
[size 16: 8069(504MB) allocated 8311 mapped]
[size 32: 18664(2333MB) allocated 18702 mapped]
[size 64: 26853(6713MB) allocated 26920 mapped]
[size 128: 23818(11909MB) allocated 23869 mapped]
[size 256: 190477(190477MB) allocated 190711 mapped]
]
E20240323 04:38:09.100278 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.66GB tinySize: 747.66KB large size: 207.66GB
Cache entries: 51020 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2495
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76037 eviction checks: 445475 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57320400 cached pages: 54437033
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 348.19MB allocated pages 57320400 mapped pages 57409412 external mapped pages 2765846
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16888(131MB) allocated 22397 mapped]
[size 4: 18827(294MB) allocated 18842 mapped]
[size 8: 21220(663MB) allocated 21287 mapped]
[size 16: 8065(504MB) allocated 8311 mapped]
[size 32: 18646(2330MB) allocated 18702 mapped]
[size 64: 26853(6713MB) allocated 26920 mapped]
[size 128: 23818(11909MB) allocated 23869 mapped]
[size 256: 190477(190477MB) allocated 190711 mapped]
]
E20240323 04:38:09.107064 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.101714 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.66GB tinySize: 747.66KB large size: 207.66GB
Cache entries: 51020 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2495
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76037 eviction checks: 445475 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57320537 cached pages: 54437033
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 347.65MB allocated pages 57320537 mapped pages 57409412 external mapped pages 2765846
[size 1: 20878(81MB) allocated 21540 mapped]
[size 2: 16887(131MB) allocated 22397 mapped]
[size 4: 18827(294MB) allocated 18842 mapped]
[size 8: 21220(663MB) allocated 21287 mapped]
[size 16: 8073(504MB) allocated 8311 mapped]
[size 32: 18646(2330MB) allocated 18702 mapped]
[size 64: 26853(6713MB) allocated 26920 mapped]
[size 128: 23818(11909MB) allocated 23869 mapped]
[size 256: 190477(190477MB) allocated 190711 mapped]
]
E20240323 04:38:09.108418 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.131866 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 373.39MB allocated pages 57313948 mapped pages 57408900 external mapped pages 2765334
[size 1: 20886(81MB) allocated 21540 mapped]
[size 2: 16894(131MB) allocated 22397 mapped]
[size 4: 18820(294MB) allocated 18842 mapped]
[size 8: 21208(662MB) allocated 21287 mapped]
[size 16: 8027(501MB) allocated 8311 mapped]
[size 32: 18621(2327MB) allocated 18702 mapped]
[size 64: 26826(6706MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190450(190450MB) allocated 190711 mapped]
]
E20240323 04:38:09.132555 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 384.19MB allocated pages 57311184 mapped pages 57409412 external mapped pages 2765846
[size 1: 20886(81MB) allocated 21540 mapped]
[size 2: 16894(131MB) allocated 22397 mapped]
[size 4: 18820(294MB) allocated 18842 mapped]
[size 8: 21208(662MB) allocated 21287 mapped]
[size 16: 8034(502MB) allocated 8311 mapped]
[size 32: 18621(2327MB) allocated 18702 mapped]
[size 64: 26826(6706MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190450(190450MB) allocated 190711 mapped]
]
E20240323 04:38:09.135048 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.63GB tinySize: 747.60KB large size: 207.63GB
Cache entries: 51016 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2499
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76041 eviction checks: 445479 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57311207 cached pages: 54429821
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 384.10MB allocated pages 57311207 mapped pages 57409412 external mapped pages 2765846
[size 1: 20887(81MB) allocated 21540 mapped]
[size 2: 16895(131MB) allocated 22397 mapped]
[size 4: 18821(294MB) allocated 18842 mapped]
[size 8: 21204(662MB) allocated 21287 mapped]
[size 16: 8042(502MB) allocated 8311 mapped]
[size 32: 18615(2326MB) allocated 18702 mapped]
[size 64: 26825(6706MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190450(190450MB) allocated 190711 mapped]
]
E20240323 04:38:09.141470 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.136657 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.63GB tinySize: 747.60KB large size: 207.63GB
Cache entries: 51016 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2499
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76041 eviction checks: 445479 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57311053 cached pages: 54429821
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 384.70MB allocated pages 57311053 mapped pages 57409412 external mapped pages 2765846
[size 1: 20885(81MB) allocated 21540 mapped]
[size 2: 16895(131MB) allocated 22397 mapped]
[size 4: 18819(294MB) allocated 18842 mapped]
[size 8: 21202(662MB) allocated 21287 mapped]
[size 16: 8013(500MB) allocated 8311 mapped]
[size 32: 18615(2326MB) allocated 18702 mapped]
[size 64: 26825(6706MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190450(190450MB) allocated 190711 mapped]
]
E20240323 04:38:09.143445 3750 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.149595 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 398.95MB allocated pages 57307406 mapped pages 57408900 external mapped pages 2765334
[size 1: 20884(81MB) allocated 21540 mapped]
[size 2: 16892(131MB) allocated 22397 mapped]
[size 4: 18821(294MB) allocated 18842 mapped]
[size 8: 21183(661MB) allocated 21287 mapped]
[size 16: 8015(500MB) allocated 8311 mapped]
[size 32: 18598(2324MB) allocated 18702 mapped]
[size 64: 26821(6705MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190440(190440MB) allocated 190711 mapped]
]
E20240323 04:38:09.152616 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.62GB tinySize: 747.47KB large size: 207.62GB
Cache entries: 51007 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2508
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76050 eviction checks: 445488 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57306799 cached pages: 54427015
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 401.32MB allocated pages 57306799 mapped pages 57408900 external mapped pages 2765334
[size 1: 20885(81MB) allocated 21540 mapped]
[size 2: 16879(131MB) allocated 22397 mapped]
[size 4: 18821(294MB) allocated 18842 mapped]
[size 8: 21167(661MB) allocated 21287 mapped]
[size 16: 8026(501MB) allocated 8311 mapped]
[size 32: 18588(2323MB) allocated 18702 mapped]
[size 64: 26821(6705MB) allocated 26920 mapped]
[size 128: 23823(11911MB) allocated 23869 mapped]
[size 256: 190440(190440MB) allocated 190711 mapped]
]
E20240323 04:38:09.158784 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.161846 3751 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 417.39MB allocated pages 57302683 mapped pages 57408900 external mapped pages 2765334
[size 1: 20883(81MB) allocated 21540 mapped]
[size 2: 16879(131MB) allocated 22397 mapped]
[size 4: 18821(294MB) allocated 18842 mapped]
[size 8: 21164(661MB) allocated 21287 mapped]
[size 16: 8046(502MB) allocated 8311 mapped]
[size 32: 18586(2323MB) allocated 18702 mapped]
[size 64: 26820(6705MB) allocated 26920 mapped]
[size 128: 23822(11911MB) allocated 23869 mapped]
[size 256: 190426(190426MB) allocated 190711 mapped]
]
E20240323 04:38:09.164783 3751 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.61GB tinySize: 747.41KB large size: 207.61GB
Cache entries: 51003 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2512
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76054 eviction checks: 445492 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57303159 cached pages: 54423096
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 415.54MB allocated pages 57303159 mapped pages 57408900 external mapped pages 2765334
[size 1: 20887(81MB) allocated 21540 mapped]
[size 2: 16883(131MB) allocated 22397 mapped]
[size 4: 18825(294MB) allocated 18842 mapped]
[size 8: 21168(661MB) allocated 21287 mapped]
[size 16: 8080(505MB) allocated 8311 mapped]
[size 32: 18586(2323MB) allocated 18702 mapped]
[size 64: 26820(6705MB) allocated 26920 mapped]
[size 128: 23822(11911MB) allocated 23869 mapped]
[size 256: 190426(190426MB) allocated 190711 mapped]
]
E20240323 04:38:09.170701 3751 MmapAllocator.cpp:415] [MEM] Mmap failed with 512 pages use MmapArena false
E20240323 04:38:09.180087 3750 MmapAllocator.cpp:404] [MEM] Data Mmap failedMemory Allocator[MMAP total capacity 219.00GB free capacity 437.51MB allocated pages 57297533 mapped pages 57408900 external mapped pages 2765334
[size 1: 20889(81MB) allocated 21540 mapped]
[size 2: 16887(131MB) allocated 22397 mapped]
[size 4: 18826(294MB) allocated 18842 mapped]
[size 8: 21171(661MB) allocated 21287 mapped]
[size 16: 8187(511MB) allocated 8311 mapped]
[size 32: 18585(2323MB) allocated 18702 mapped]
[size 64: 26816(6704MB) allocated 26920 mapped]
[size 128: 23817(11908MB) allocated 23869 mapped]
[size 256: 190401(190401MB) allocated 190711 mapped]
]
E20240323 04:38:09.183209 3751 Exceptions.h:69] Line: /app/presto-native-execution/velox/velox/common/memory/MemoryPool.cpp:1191, Function:handleAllocationFailure, Expression: allocateContiguous failed with 512 pages from Memory Pool[op.10.0.0.Aggregation LEAF root[20240323_043257_00076_tjfjm_72] parent[node.10] MMAP track-usage thread-safe]<max capacity 208.00GB unlimited capacity used 5.19GB available 7.50MB reservation [used 5.19GB, reserved 5.20GB, min 0B] counters [allocs 41, frees 15, reserves 0, releases 0, collisions 0])> Mmap failed with 512 pages use MmapArena false Failed to evict from cache state: AsyncDataCache:
Cache size: 207.60GB tinySize: 747.37KB large size: 207.59GB
Cache entries: 51000 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2515
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76057 eviction checks: 445495 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57301058 cached pages: 54419726
, Source: RUNTIME, ErrorCode: MEM_ALLOC_ERROR
E20240323 04:38:09.183156 3750 MmapAllocator.cpp:406] [MEM] AsyncDataCache:
Cache size: 207.58GB tinySize: 747.31KB large size: 207.58GB
Cache entries: 50996 read pins: 160 write pins: 0 pinned shared: 586.47MB pinned exclusive: 0B
num write wait: 7416 empty entries: 2519
Cache access miss: 127057 hit: 513945 hit bytes: 1.63TB eviction: 76061 eviction checks: 445499 aged out: 0
Prefetch entries: 22 bytes: 115.99MB
Alloc Megaclocks 1096275
Allocated pages: 57298267 cached pages: 54415698
Backing: Memory Allocator[MMAP total capacity 219.00GB free capacity 434.64MB allocated pages 57298267 mapped pages 57408900 external mapped pages 2765334
[size 1: 20891(81MB) allocated 21540 mapped]
[size 2: 16889(131MB) allocated 22397 mapped]
[size 4: 18828(294MB) allocated 18842 mapped]
[size 8: 21173(661MB) allocated 21287 mapped]
[size 16: 8223(513MB) allocated 8311 mapped]
[size 32: 18585(2323MB) allocated 18702 mapped]
[size 64: 26816(6704MB) allocated 26920 mapped]
[size 128: 23817(11908MB) allocated 23869 mapped]
[size 256: 190401(190401MB) allocated 190711 mapped]
]
PR resolved the issue.
Bug description
Under certain conditions memory cannot be allocated by the MemoryAllocator. This occasionally on clusters running TPCDS queries.
Prestissimo does not explicitly set the option
useMmapArena
and the default value offalse
is used by Velox. As a result, contiguous memory requests use themmap
system call.The code in
allocateContiguousImpl
performing the allocation may callmmap
.If the
mmap
call fails then data will be set toMAP_FAILED (-1)
anderrno
is set. The code has a TODO comment to handle MAP_FAILED which == -1.The proposed solution to prevent the SEGV and return the error (fail query with insufficient resources) would be to check for MAP_FAILED
System information
Host with 122 GiB memory available. Async data cache enabled by default.
system-memory-gb=108 query-memory-gb=102 query.max-memory-per-node=102GB memory-arbitrator-kind=SHARED task.max-drivers-per-task=16
Relevant logs