Granulate / gprofiler

gProfiler is a system-wide profiler, combining multiple sampling profilers to produce unified visualization of what your CPU is spending time on.
https://profiler.granulate.io
Apache License 2.0
746 stars 54 forks source link

perf: Optionally utilize buildid cache #707

Open Jongy opened 1 year ago

Jongy commented 1 year ago

perf has a feature called buildid cache - read about it here. The idea is that perf record will copy aside all binaries mapped in profiled apps, so even if the apps go down by the time we run perf script, we have the binaries and are able to perform symbolication.

We had it in the past and it was removed in #113. It affects when profiling short-lived containers - similarly to the problem described in this ticket. If we're profiling a container and it goes down before we run perf script, then perf script is unable to access the container files, and it cannot symbolicate, and we end up with unknown frames (or frames which are just the DSO name). In a system with very short-lived containers, we end up missing most of the symbols :/

I suggest we allow utilizing buildid cache as an optional feature (e.g gprofiler --perf-enable-buildid-cache), also allow limiting its size somehow (if perf allows it then via perf, otherwise by continuously monitoring it and removing excessive files?). We need to modify perf record to collect DSOs during record, and perf script to use them during scripting.

See this Linux commit for the reason it's disabled by default in switch-output mode (that we added in #113) and how to enable it anyway. Further usage instructions are found in Linux docs.