search

bencheeorg / benchee

Easy and extensible benchmarking in Elixir providing you with lots of statistics!
MIT License
1.42k stars 66 forks source link

Add reduction counting #307

Closed devonestes closed 4 years ago

devonestes commented 5 years ago

This has been sitting on my machine like half done for a month or so, and I finally finished it!

This adds reduction counting, which behaves almost exactly like memory measurement. This should finally allow folks to do really high-quality performance testing that won't be affected by all the other random stuff that can affect clock time.

It's still a little rough, but it works!

OvermindDL1 commented 5 years ago

Oh this looks cool! :-)

The CI failures just seem to be credo failing strict checks because of duplicate code that could be refactored out into a helper module between the different areas it looks like, as well as a dialyzer failure due to a spec needing fixing it seems, and 2 bad tests?

PragTob commented 5 years ago

FYI I haven't been ignoring this just put on the back burner as the whole job hunt thing was quite stressful (and I figured Devon knew/guessed as much). That is (almost) at an end now so I hope to get to this this week.

devonestes commented 4 years ago

@PragTob Hey, so I'm finally back on this - is there anything that you think we still need before this can hit master? Of course we need to add this to the other formatters, but that'll happen once this is merged, ya?

PragTob commented 4 years ago

@devonestes Hola! Sorry haven't given benchee the love it deserved in a long time :) Should be fine let me double check :)

PragTob commented 4 years ago

Looking good to me, there's just the one more thing: https://github.com/bencheeorg/benchee/pull/307#discussion_r374303192