search

LeventErkok / sbv

SMT Based Verification in Haskell. Express properties about Haskell programs and automatically prove them using SMT solvers.
https://github.com/LeventErkok/sbv
Other
240 stars 33 forks source link

Move benchmarks to tasty-bench #608

Closed LeventErkok closed 2 years ago

LeventErkok commented 2 years ago

Looks like basement is broken (https://github.com/haskell-foundation/foundation/issues/554) and thus gauge no longer compiles with GHC 9.2.1. This breaks our benchmark suite.

It appears tasty-bench (https://hackage.haskell.org/package/tasty-bench) is a new alternative, with much fewer dependencies than Gauge. It also fits well with SBV: We already use tasty in our test-suite, so it looks like a good alternative.

We should move the benchmarks to tasty-bench.

@doyougnu Is this something you can handle? Not urgent of course, but since you're the original developer of the SBV-benchsuite, this might be easy for you to do. As I understand, tasty-bench provides a "compatible" API with gauge, so porting may not be that difficult.

doyougnu commented 2 years ago

@LeventErkok yup I can take this on. It'll also be a good project to get back into sbv

LeventErkok commented 2 years ago

@doyougnu Thanks! Much appreciated.

Bodigrim commented 2 years ago

FWIW you can use a snippet at https://hackage.haskell.org/package/tasty-bench-0.3.1/docs/Test-Tasty-Bench.html#t:CsvPath to set default CSV path and timeout.

LeventErkok commented 2 years ago

@Bodigrim Thanks! I'm sure Jeff (@doyougnu) will take a look at that..

doyougnu commented 2 years ago

Ah thanks for the pointer! I actually implemented this in the benchmark suite to work around gauge, so it will be nice to offload this code to tasty-bench. From browsing the docs I think using tasty-bench will significantly simplify the benchmark suite.