I'm using bytehound to try to pinpoint what in my program is allocating or reserving the large amounts of memory that cause my program to crash. Previously, #21 and #18 recommended the "squeeze" subcommand (which is now "strip"), and that was working for me for some time on files of size 700GiB+ before I began setting MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 and occasionally MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=100000 or thereabouts when running my program through bytehound.
I have 48 GiB of RAM on my system. An 11 GiB data file was generated when I ran my program with bytehound and set MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=50000; I killed the program via SIGINT when it reached 93% memory usage. Running this file through bytehound server doesn't work due to OOM. Surprisingly, running this file through bytehound strip is also causing an OOM. Via top I can watch the virtual memory space increase to 111GiB whereas the program only reached 84GiB virtual.
Do I need to set the same parameters when running strip? Or is there some other reason that it runs out of memory?
I'm using bytehound to try to pinpoint what in my program is allocating or reserving the large amounts of memory that cause my program to crash. Previously, #21 and #18 recommended the "squeeze" subcommand (which is now "strip"), and that was working for me for some time on files of size 700GiB+ before I began setting
MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1and occasionallyMEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=100000or thereabouts when running my program through bytehound.I have 48 GiB of RAM on my system. An 11 GiB data file was generated when I ran my program with bytehound and set
MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=50000; I killed the program via SIGINT when it reached 93% memory usage. Running this file throughbytehound serverdoesn't work due to OOM. Surprisingly, running this file throughbytehound stripis also causing an OOM. ViatopI can watch the virtual memory space increase to 111GiB whereas the program only reached 84GiB virtual.Do I need to set the same parameters when running
strip? Or is there some other reason that it runs out of memory?