Granulate / gprofiler

gProfiler is a system-wide profiler, combining multiple sampling profilers to produce unified visualization of what your CPU is spending time on.
https://profiler.granulate.io
Apache License 2.0
746 stars 54 forks source link

Add continuous collection of profiles in java profiler #730

Open marcin-ol opened 1 year ago

marcin-ol commented 1 year ago

Add more accurate collection of java application profiles using async-profiler's dump command.

Description

This PR changes the way that async-profiler is invoked. From now on gprofiler starts ap once and periodically issues dump command to retrieve the collapsed profiles.

Related Issue

569

Motivation and Context

More accurate profiling and better performance due to having async-profiler loaded in process.

How Has This Been Tested?

Whole collection of java tests in repository was executed against this change.

Checklist:

marcin-ol commented 1 year ago

Several things need addressing (and the solution is underway):

  • [x] adjust output for cumulative frame count - async-profiler doesn't reset it, frame counts are increasing,
  • [ ] account for dump skew - time between last dump output and the next call to snapshot().

(these are no longer needed, handled in async-profiler and in gProfiler, respectively).

marcin-ol commented 1 year ago

Java profiling is now truly continuous thanks to async-profiler timeout recycle functionality: https://github.com/Granulate/async-profiler/pull/6 .

marcin-ol commented 1 year ago

Last commit got pushed too early. Adjustment is correct in principle, but needs to take multiple profilers into account.

marcin-ol commented 1 year ago

Output from profiler in continuous mode is now equivalent to output produced by profiler on master. Frame counts now cover only the snapshot window.

marcin-ol commented 1 year ago

Please merge master & update async-profiler to the latest revision used in master, I'll get to reviewing this soon.

Done.

marcin-ol commented 1 year ago

Added resettrace flag to async-profiler, which resets frame counts after dump. This removes need for adjusting collected stacks, mentioned in https://github.com/Granulate/gprofiler/pull/730#issuecomment-1476824409.

marcin-ol commented 1 year ago

Also, adjusting for time skew is not needed. gProfiler does it automatically, when merging collected stacks from runtime profilers with system profiler.

marcin-ol commented 1 year ago

One remaining task:

marcin-ol commented 1 year ago

Implemented smarter error handling in async-profiler with dumpactive flag. Command dump will now report error if profiler is stopped, letting java profiler know that it needs to restart profiling. This removes need for status calls, making java profiling more fluent.