google / pprof

pprof is a tool for visualization and analysis of profiling data
Apache License 2.0
7.83k stars 604 forks source link

How to implement `/pprof/symbol` API for RelWithDebInfo ELF . #890

Open IronsDu opened 2 weeks ago

IronsDu commented 2 weeks ago

Please answer these questions before submitting your issue. Thanks!

What version of pprof are you using?

If you are using pprof via go tool pprof, what's your go env output? If you run pprof from GitHub, what's the Git revision?

Git revision: da1f7e9f2b251de696b28108b5a2b1c0bbce44ed

What operating system and processor architecture are you using?

Ubuntu 24.04(WSL) 、x86-64

What did you do?

If possible, provide a recipe for reproducing the error. Attaching a profile you are trying to analyze is good.

I would like to implement a heap profiler of the program remotely via pprof.

What did you expect to see?

I'd like to be able to see the correct and complete Flame Graph. Even if the program is the RelWithDebInfo version. Even if it's an inline function, I can see the complete call chain.

What did you see instead?

When using the RelWithDebInfo version of the program, opening two threads in which each loops through a different chain of calls: a->b->c , or b->c, yields a flame map with only one line: probably clone_thread -> c.

This is because I'm getting a pprof request/pprof/symbol that have the c function in the addr, but not have a. I'm not sure if this is what I'm looking for, b function's address. My /pprof/symbol only returns a function name for each addr.

But I see that the doLocalSymbolize logic in the pprof source gets the full call chain for an addr. and I added some logging to the pprof and I see that the same addr may produce multiple different function names . Is that the difference? But it works correctly if I use the Debug version of the program.

How do I implement the /pprof/symbol interface? Should I return the addr's call stack?

IronsDu commented 1 week ago

https://github.com/google/pprof/blob/main/internal/symbolizer/symbolizer.go#L232 generate Line array from stack (inline function). But, when use remote symbol, https://github.com/google/pprof/blob/main/internal/symbolz/symbolz.go#L178 only generate one function for ever addr.

Bug, the origin pprof (in gperftools) allow multi function for one addr , please see : https://github.com/gperftools/gperftools/blob/master/src/pprof#L3508