search

grafana / pyroscope-rs

Pyroscope Profiler for Rust. Profile your Rust applications.
Apache License 2.0
133 stars 22 forks source link

Profiling cycle #19

Closed drahnr closed 2 years ago

drahnr commented 2 years ago

Currently all profiles are gathered only 10s and this is hardcoded.

In pyroscope we see the same pattern while the sampling period is 113 Hz image

I would have expected to see a full timeline with the 113 bars in the span of 1 second, rather than the average over 10s with a sample rate of 113 Hz.

Is this intended?

omarabid commented 2 years ago

@drahnr I think yes. To be honest, I'm not quite sure what these bars mean. @petethepig might help here?

petethepig commented 2 years ago

@drahnr Yeah, this is intended. The idea with the timeline is that it gives you a rough idea of where CPU time was spent. For example, on the screenshot you linked the second bar is little higher — that means CPU was a little more active at that point.

This is due to the fact that when we upload data to pyroscope we aggregate all samples in 10-second intervals and as a result lose some precision.

drahnr commented 2 years ago

My expectation was, that the data sampled is compressed, but not coalesced. The backend does provide the same functionality already, so I don't see the rationale to do this on the client side? The amount of data sent is small~ish afaiu.

For our use case, CPU usage highly depends on a certain stage, so a sample period of 10 seconds is significantly too long to see meaningful items in the flamegraphs since the majority is consumed be async related helper functions, it's also too long to see spikes due to certain functionality.

I'd be interested in making this configurable and I have a PR ready.

omarabid commented 2 years ago

Issue moved to pyroscope-io/pyroscope #900 via ZenHub

drahnr commented 2 years ago

Yikes, just created on as well