apache / brpc

brpc is an Industrial-grade RPC framework using C++ Language, which is often used in high performance system such as Search, Storage, Machine learning, Advertisement, Recommendation etc. "brpc" means "better RPC".
https://brpc.apache.org
Apache License 2.0
16.56k stars 3.98k forks source link

support jemalloc profiler #2737

Closed GreateCode closed 2 months ago

GreateCode commented 3 months ago

What problem does this PR solve?

Issue Number: resolve #1962

Problem Summary: 支持jemalloc profiler

What is changed and the side effects?

Changed: 增加jemalloc mallctl控制profile。

用法:

  1. 编译jemalloc需--enable-prof
  2. 启动进程:MALLOC_CONF="prof:true" LD_PRELOAD=/xxx/lib/libjemalloc.so ./bin/test_server
  3. 可使用FLAGS_je_prof_active在进程running期间开启/关闭采样;
  4. 改变FLAGS_je_prof_dump会dump heap文件;
  5. FLAGS_je_prof_reset可清理已采样数据和重置prof选项,并且动态设置采样率,默认2^19B;
  6. 若要做memory leak,需MALLOC_CONF="prof:true,prof_leak:true,prof_final:true",进程退出时生成heap文件

注:每次dump的都是从采样至今的所有数据,除非reset,接来下dump的是从reset至今的所有数据,方便做diff。

样例: image image image

Side effects:


Check List:

wwbmmm commented 3 months ago

感谢贡献! 能否顺便更新一下使用文档?

GreateCode commented 3 months ago

好的,我近期更新下

GreateCode commented 3 months ago

感谢贡献! 能否顺便更新一下使用文档?

done

yanglimingcn commented 3 months ago

bazel编译,也是动态链接libjemalloc.so库吗

GreateCode commented 3 months ago

bazel编译,也是动态链接libjemalloc.so库吗

不强制,和编译方式无关,只需profiler时能找到mallctl即可。 所以静态链接jemalloc时,不用LD_PRELOAD=/xxx/lib/libjemalloc.so,直接MALLOC_CONF="prof:true" bin/xxx。

yanglimingcn commented 3 months ago

LGTM

GreateCode commented 3 months ago

补充下,用静态方式链接jemalloc,进程加载其他so库比如dlopen("/xxx/libjvm.so", xx),so里函数申请的内存,可能不会被profiler采样到。

wwbmmm commented 3 months ago

LGTM

GreateCode commented 3 months ago

可以集成到内置服务heap页面吗?

尝试了,放弃了。。 你帮忙改下?

GreateCode commented 2 months ago

@wwbmmm @yanglimingcn @chenBright 接口新增了jemalloc stats信息,帮助查看memory细节如 各arena,metadata等,麻烦看下还有问题吗?谢谢~

image
chenBright commented 2 months ago

MemoryService也支持一下jemalloc stats?

GreateCode commented 2 months ago

MemoryService也支持一下jemalloc stats?

done

yanglimingcn commented 2 months ago

这个PR还有别的问题吗 @chenBright