Disclaimer
By default tcmalloc is responsible for a small amount of used memory in kqp, dq and shards. The most part of memory is used by MKQL allocator. To force all the memory to be allocated through tcmalloc - do build with the -DPROFILE_MEMORY_ALLOCATIONS flag.
Allow to dump the full state of tcmalloc heap into logs:
enable the tcmalloc runtime fork support,
set soft-limit via config or via runtime Immediate Control Board: TCMallocControls.MemoryLimit,
set the callback on hitting soft-limit: the callback forks the process, the child sends SIGSTOP to parent, prints its tcmalloc heap, sends SIGCONT.
This trick should work almost always since the parent's full memory state is shared for CoW with child. And since we stop parent's process there should be no extreme memory consumption while printing to logs.
Disclaimer By default tcmalloc is responsible for a small amount of used memory in kqp, dq and shards. The most part of memory is used by MKQL allocator. To force all the memory to be allocated through tcmalloc - do build with the
-DPROFILE_MEMORY_ALLOCATIONSflag.Allow to dump the full state of tcmalloc heap into logs:
TCMallocControls.MemoryLimit,This trick should work almost always since the parent's full memory state is shared for CoW with child. And since we stop parent's process there should be no extreme memory consumption while printing to logs.