火焰图

[WALL-E]

采样源码

@openresty-systemtap-toolkit/sample-bt -p `pgrep redis-server` -t 5 -u -d

systemtap

probe begin {
    warn(sprintf("Tracing %d (/root/redis-3.2.8/src/redis-server) in user-space only...\n", target()))
}

global bts;
global quit = 0;

probe timer.profile {
    if (pid() == target()) {
        if (!quit) {
            bts[ubacktrace()] <<< 1;

        } else {

            foreach (bt in bts- limit 1024) {
                print_ustack(bt);
                printf("\t%d\n", @count(bts[bt]));
            }

            exit()
        }
    }
}

probe timer.s(5) {
    nstacks = 0
    foreach (bt in bts limit 1) {
        nstacks++
    }

    if (nstacks == 0) {
        warn("No backtraces found. Quitting now...\n")
        exit()

    } else {
        warn("Time's up. Quitting now...(it may take a while)\n")
        quit = 1
    }
}

[WALL-E]

获取采样数据

@openresty-systemtap-toolkit/sample-bt -p `pgrep redis-server` -t 5 -u > a.bt

a.bt

 0x7f96f75e8c3d : __open_nocancel+0x24/0x57 [/usr/lib64/libpthread-2.17.so]
 0x42cd9d : zmalloc_get_rss+0x58/0x159 [/root/redis-3.2.8/src/redis-server]
 0x422a15 : serverCron+0xff/0x84c [/root/redis-3.2.8/src/redis-server]
 0x41d394 : processTimeEvents+0x1a1/0x1ff [/root/redis-3.2.8/src/redis-server]
 0x41d6b7 : aeProcessEvents+0x2c5/0x2cd [/root/redis-3.2.8/src/redis-server]
 0x41d7d0 : aeMain+0x48/0x55 [/root/redis-3.2.8/src/redis-server]
 0x429f3a : main+0x6ec/0x707 [/root/redis-3.2.8/src/redis-server]
 0x7f96f723ab35 : __libc_start_main+0xf5/0x1c0 [/usr/lib64/libc-2.17.so]
 0x4192a9 : _start+0x29/0x30 [/root/redis-3.2.8/src/redis-server]
    1
 0x7f96f7310d13 : __epoll_wait_nocancel+0x2a/0x57 [/usr/lib64/libc-2.17.so]
 0x41c9d6 : aeApiPoll+0x85/0x15f [/root/redis-3.2.8/src/redis-server]
 0x41d59c : aeProcessEvents+0x1aa/0x2cd [/root/redis-3.2.8/src/redis-server]
 0x41d7d0 : aeMain+0x48/0x55 [/root/redis-3.2.8/src/redis-server]
 0x429f3a : main+0x6ec/0x707 [/root/redis-3.2.8/src/redis-server]
 0x7f96f723ab35 : __libc_start_main+0xf5/0x1c0 [/usr/lib64/libc-2.17.so]
 0x4192a9 : _start+0x29/0x30 [/root/redis-3.2.8/src/redis-server]
    1
 0x7f96f75e849d : __read_nocancel+0x24/0x57 [/usr/lib64/libpthread-2.17.so]
 0x42cdc9 : zmalloc_get_rss+0x84/0x159 [/root/redis-3.2.8/src/redis-server]
 0x422a15 : serverCron+0xff/0x84c [/root/redis-3.2.8/src/redis-server]
 0x41d394 : processTimeEvents+0x1a1/0x1ff [/root/redis-3.2.8/src/redis-server]
 0x41d6b7 : aeProcessEvents+0x2c5/0x2cd [/root/redis-3.2.8/src/redis-server]
 0x41d7d0 : aeMain+0x48/0x55 [/root/redis-3.2.8/src/redis-server]
 0x429f3a : main+0x6ec/0x707 [/root/redis-3.2.8/src/redis-server]
 0x7f96f723ab35 : __libc_start_main+0xf5/0x1c0 [/usr/lib64/libc-2.17.so]
 0x4192a9 : _start+0x29/0x30 [/root/redis-3.2.8/src/redis-server]
    1

[WALL-E]

合并采样数据

@FlameGraph/stackcollapse-stap.pl /tmp/a.bt  > /tmp/a.cbt

/tmp/a.cbt

_start;__libc_start_main;main;aeMain;aeProcessEvents;aeApiPoll;__epoll_wait_nocancel 1
_start;__libc_start_main;main;aeMain;aeProcessEvents;processTimeEvents;serverCron;zmalloc_get_rss;__open_nocancel 1
_start;__libc_start_main;main;aeMain;aeProcessEvents;processTimeEvents;serverCron;zmalloc_get_rss;__read_nocancel 1

[WALL-E]

生成svg

@FlameGraph/flamegraph.pl /tmp/a.cbt > /tmp/a.svg

https://github.com/WALL-E/systemtap-practice/blob/master/FlameGraph/a.svg

[WALL-E]

火焰图只是一种统计数据的展现方式，本身并没有什么神奇或是高深的地方。 生成一个火焰图很容易，难点是从火焰图中发现问题，并且能够给出较为合理的解释，再进一步给出优化方案。