So, we provide some protection. In general, there are two means of protection: addr_validate and blocklist. addr_validate ensures that we do not try to access unreadable addresses, and blocklist helps us exclude stack frames belonging to specified shared libraries.
Issue
But the commit above introduces a problem, here is a minimal reproduction:
use pprof::ProfilerGuardBuilder;
use std::fs::File;
fn main() {
let g = ProfilerGuardBuilder::default()
.blocklist(&["libc", "libgcc", "pthread", "vdso"])
.build()
.unwrap();
workload();
g.report()
.build()
.unwrap()
.flamegraph(File::create("/tmp/r.svg").unwrap())
.unwrap();
}
fn workload() -> Vec<i32> {
let mut rs = vec![];
for _ in 0..100_000_000 {
let mut v = vec![];
for n in 100_000_000..0 {
v.push(n);
}
v.sort();
rs.push(v.iter().sum());
}
rs
}
To avoid deadlocks, we will block shared libraries like libc, pthread, etc, but this causes an additional problem. Let's see the generated svg:
It only contains samples inside the signal handler, all stack frames before the signal frame are lost.
Reason
The code in the RED BOX is newly introduced by the commit above, with frame-pointer based stacktracing. Its purpose is to do extra protection in the callback of each stack frame. But when doing stacktraces with backtrace-rs, there are some magical dependencies on libc in order to handle signal frames on the stack. So they are blocked due to the existence of blocklist.
But why does it works well with frame-pointer feature? Because frame-pointer based stacktracing starts from ucontext provided by OS kernel, which is an argument of perf_signal_handler. ucontext holds the top stack frame before SIGPROF triggered, so we don't have to deal with signal frames at all.
Solve
Only do per-frame-blocklist-check when frame-pointer is enabled. We will get the correct flamegraph:
Signed-off-by: mornyx mornyx.z@gmail.com
Related Issues
Background
We introduced frame-pointer based stacktrace in https://github.com/tikv/pprof-rs/commit/567c5d01c9bd8deb9ccbe5515122d5aa9faef0ad. But we cannot guarantee that all shared libraries have frame pointers enabled, and performing stacktracing directly based on frame pointers is dangerous and may cause crash.
So, we provide some protection. In general, there are two means of protection:
addr_validateandblocklist.addr_validateensures that we do not try to access unreadable addresses, andblocklisthelps us exclude stack frames belonging to specified shared libraries.Issue
But the commit above introduces a problem, here is a minimal reproduction:
To avoid deadlocks, we will block shared libraries like
libc,pthread, etc, but this causes an additional problem. Let's see the generated svg:It only contains samples inside the signal handler, all stack frames before the signal frame are lost.
Reason
The code in the RED BOX is newly introduced by the commit above, with frame-pointer based stacktracing. Its purpose is to do extra protection in the callback of each stack frame. But when doing stacktraces with
backtrace-rs, there are some magical dependencies onlibcin order to handle signal frames on the stack. So they are blocked due to the existence of blocklist.But why does it works well with
frame-pointerfeature? Because frame-pointer based stacktracing starts fromucontextprovided by OS kernel, which is an argument ofperf_signal_handler.ucontextholds the top stack frame before SIGPROF triggered, so we don't have to deal with signal frames at all.Solve
Only do per-frame-blocklist-check when
frame-pointeris enabled. We will get the correct flamegraph: