Profile with rayon?

[alexkirsz]

Hey,

I'm using rayon to parallelize some data processing and I noticed that the profile_fn and profile_section instances don't show when called inside a rayon thread. Is there a way to record them?

Cheers, Alex

[That3Percent]

There is a way to get at this information and I worked out how to do it once with a flamegraph profiler that I used before writing firestorm, but the code is now missing.

The basic strategy if I recall correctly was to use a rayon threadpool which would allow you to control thread termination by dropping the pool. Once you have that you can have each thread save a separate flamegraph on drop.

That's far from ideal, obviously. One problem is that it's difficult to correlate events across multiple threads or show idle time in a meaningful way. What would be better is if you could save a single flamegraph that showed data from multiple threads at the same time. I think this is possible, but may require some upstream changes from inferno.

In any case I think it would be a reasonable goal to have support builtin to firestorm first to support timelines across threads, and then to even add explicit support for rayon since that's such a popular choice.

PRs welcome.

[cmpute]

I used easy_profiler in C++, and it can capture the performance in multi-threaded environment. Maybe this could be a reference?

[ColonelThirtyTwo]

Getting a similar issue even without Rayon.

@That3Percent Are you saying that this crate can't profile across threads? That seems like a big limitation that you should put in the readme.