bencheeorg / benchee

Easy and extensible benchmarking in Elixir providing you with lots of statistics!
MIT License
1.42k stars 66 forks source link

More generic profiler support (eflambe in particular) #440

Open PragTob opened 4 days ago

PragTob commented 4 days ago

I saw a cool talk at Code BEAM EU about performance optimizations and the 1 billion row challenge by @rajrajhans and he used eflambe for profiling.

So the question was... why not add support for more profilers? Right now we only support built-in profilers.

In true benchee mannerism though I'd want it to be more generic:

likely similiar to the interface of the built-in profilers which we already support: https://hexdocs.pm/mix/Mix.Tasks.Profile.Tprof.html

I'd expect that interface to be quite simple, but I'm not looking deeply into it just now :) Here is where/how we currently use them: https://github.com/bencheeorg/benchee/blob/main/lib/benchee/profile.ex#L109

https://github.com/bencheeorg/benchee/blob/main/lib/benchee/benchmark/collect/profile.ex

It boils down to:

    return_value = profiler_module.profile(function, profiler_opts)

With that we could also remove our reliance on the mix tasks (I'm unsure how much benefit they give us over calling the profilers themselves).