bencheeorg / benchee

Easy and extensible benchmarking in Elixir providing you with lots of statistics!
MIT License
1.41k stars 66 forks source link

Warn if the benchmarking function is evaluating #355

Closed josevalim closed 2 years ago

josevalim commented 2 years ago

With Mix.install/2, there is an increased odd that someone will invoke Benchee from IEx or from a Livebook. The problem is that evaluated functions are much slower. I suggest emit warnings if the benchmark functions are evaluating.

You can check if a function is evaluating like this:

:erlang.fun_info(fun, :module) == {:module, :erl_eval}

The warning could look like this:

warning: the benchmark "foobar" is using an evaluated function. Evaluated functions perform slower than compiled functions. Consider moving all of the Benchee caller to a function in a module and invoking the `Mod.fun()` instead.

Thank you for the project and congrats on v1.0!

PragTob commented 2 years ago

Thanks José!

That sounds like a great improvement to have, I always wanted to have it but didn't know how. So, thanks for providing the how!

IMG_20220130_140846_Bokeh

PragTob commented 2 years ago

fixed in the way of #358, revived as #365