google / tcmalloc

Apache License 2.0
4.41k stars 480 forks source link

tcmalloc::MallocExtension::PerCpuCachesActive() returning false #251

Closed lano1106 closed 3 months ago

lano1106 commented 3 months ago

I have this simple program:

#include <iostream>
#include "tcmalloc/malloc_extension.h"

int main(int argc, char *argv[])
{
    char *ptr = new char;

    *ptr = 'a';
    std::cout << *ptr << " PerCpuCachesActive:"
              << tcmalloc::MallocExtension::PerCpuCachesActive() << '\n';
    delete ptr;
    std::cout << tcmalloc::MallocExtension::GetStats() << '\n';
    return 0;
}

compiled with: g++ -g -std=c++26 -I/home/lano1106/dev/tcmalloc tcmalloc_new_test.cpp -L/home/lano1106/dev/tcmalloc/bazel-bin/tcmalloc/ -ltcmalloc_new

the so file has been created by adding to tcmalloc/BUILD

cc_binary(
    name = "libtcmalloc_new.so",
    deps = [":tcmalloc"],
    linkshared = 1,
    copts = TCMALLOC_DEFAULT_COPTS,
)

I am trying to understand how to enable the PerCpuCaches...

My kernel is 6.10.3. I have validated that the rseq syscall is available:

     InputThread-1115    [004] ..... 582794.342439: rseq_update: cpu_id=4 node_id=0 mm_cid=0
             cat-994707  [007] ..... 582794.342456: rseq_update: cpu_id=7 node_id=0 mm_cid=0
  xfce4-terminal-1312    [000] ..... 582794.342471: rseq_update: cpu_id=0 node_id=0 mm_cid=0
            Xorg-1096    [009] ..... 582794.342480: rseq_update: cpu_id=9 node_id=0 mm_cid=2
             cat-994707  [007] ..... 582794.342486: rseq_update: cpu_id=7 node_id=0 mm_cid=0
  xfce4-terminal-1312    [000] ..... 582794.342501: rseq_update: cpu_id=0 node_id=0 mm_cid=0
  xfce4-terminal-1312    [000] ..... 582794.342514: rseq_update: cpu_id=0 node_id=0 mm_cid=0
        WSThread-918300  [002] ..... 582794.342516: rseq_update: cpu_id=2 node_id=0 mm_cid=3
             cat-994707  [007] ..... 582794.342526: rseq_update: cpu_id=7 node_id=0 mm_cid=0
    book_monitor-918291  [008] ..... 582794.342538: rseq_update: cpu_id=8 node_id=0 mm_cid=2
  xfce4-terminal-1312    [000] ..... 582794.342552: rseq_update: cpu_id=0 node_id=0 mm_cid=0
             cat-994707  [007] ..... 582794.342561: rseq_update: cpu_id=7 node_id=0 mm_cid=0
$ bazel run //tcmalloc:cpu_cache_activate_test
INFO: Analyzed target //tcmalloc:cpu_cache_activate_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //tcmalloc:cpu_cache_activate_test up-to-date:
  bazel-bin/tcmalloc/cpu_cache_activate_test
INFO: Elapsed time: 0.097s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh tcmalloc/cpu_cache_activate_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tcmalloc:cpu_cache_activate_test
-----------------------------------------------------------------------------
Running main() from gmock_main.cc
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CpuCacheActivateTest
[ RUN      ] CpuCacheActivateTest.GlobalInstance
[       OK ] CpuCacheActivateTest.GlobalInstance (0 ms)
[----------] 1 test from CpuCacheActivateTest (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 1 test.

next step is to modify the tcmalloc/testing:hello_main to see if a pure bazel exec is behaving differently... I might also start to pepper tcmalloc code with printf to track the issue but before going through all these loops and hoops, I am just checking with you if anyone has an idea why my percpu caches are not enabled...

lano1106 commented 3 months ago
$ bazel run tcmalloc/testing:hello_main
INFO: Analyzed target //tcmalloc/testing:hello_main (0 packages loaded, 2 targets configured).
INFO: Found 1 target...
Target //tcmalloc/testing:hello_main up-to-date:
  bazel-bin/tcmalloc/testing/hello_main
INFO: Elapsed time: 3.929s, Critical Path: 3.79s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
INFO: Running command line: bazel-bin/tcmalloc/testing/hello_main
Current heap size = 73728 bytes
hello world!
new'd 1073741824 bytes at 0x7174c0000000
Current heap size = 1073816576 bytes
malloc'd 1073741824 bytes at 0x717500000000
Current heap size = 2147558400 bytes
 PerCpuCachesActive:0
lano1106 commented 3 months ago
$ GLIBC_TUNABLES=glibc.pthread.rseq=0 ./a.out
a PerCpuCachesActive:1

I have found this idea in #144

I have glibc v2.40 on my system

ckennelly commented 3 months ago

Dupe of #144