search

purescript / purescript-free

Free monads, Cofree comonads, Yoneda and Coyoneda functors, and the Trampoline monad.
BSD 3-Clause "New" or "Revised" License
93 stars 27 forks source link

Update benchmarks #103

Closed thomashoneyman closed 4 years ago

thomashoneyman commented 4 years ago

This PR updates the benchmarks so they can be run on the current (v5.2.0) and previous (v0.6.1) implementations. The old (v0.6.1) implementation suffers from a stack overflow in the left-bind benchmark, so for the time being I've simply disabled it. Open to suggestions if anyone knows how to fix that!

This PR supersedes #88. It also includes commands in the package.json file to build and run the benchmarks. Finally, it compiles (but does not run) the benchmarks as part of CI, just so they don't get so horribly out of date again in the future.

As part of this change, I've moved the current v5.2.0 implementation of Free / Trampoline into the benchmarks directory. That way we can use it as a reference when testing any future performance changes, as we're not exactly going to return to the v0.6.1 design -- v5.2.0 is a better reference for future benchmarks.

This PR is a first step towards testing a new implementation for Free which is significantly faster in the Halogen Hooks benchmarks: https://github.com/thomashoneyman/purescript-halogen-hooks/pull/61

Also closes #47.

thomashoneyman commented 4 years ago

These are the charts generated with https://harry.garrood.me/purescript-benchotron-svg-renderer for the four benchmarks. The left-bind benchmark with large inputs doesn't contain the v0.6.1 code because it has a stack overflow.

right-bind-small

right-bind-large

left-bind-small left-bind-large

thomashoneyman commented 4 years ago

(@safareli -- as you worked on #88, do you have any comments on this?)

safareli commented 4 years ago

Unfortunately I can't review this as I'm busy for this month. Thanks for working on this tho!

thomashoneyman commented 4 years ago

Unless a maintainer objects, I'll go ahead and merge this within the next few days given that it doesn't affect the library internals and re-enables a very useful set of benchmarks. The disabled test can be re-enabled in the future if necessary.

garyb commented 4 years ago

Sounds good to me