search

IntersectMBO / lsm-tree

A Haskell library for on-disk tables based on LSM-trees
Apache License 2.0
24 stars 7 forks source link

pipelined mode for WP8 benchmark #280

Closed dcoutts closed 1 month ago

dcoutts commented 1 month ago

Add the pipelined mode to the WP8 benchmark.

This is a bit awkward because to work with the real implementation it requires the duplicate action, which isn't implemented yet. So we can't get performance results for this yet.

I've tested the pipelined mode with the model implementation and that works fine, but I don't want to include that in the benchmark code because it pulls in too many other dependencies.

So to test the pipelined (and sequential) versions I've included a dynamic check that the results are the same as a simple pure reference function. This is enabled by --check. Hopefully the presence of the observation/output doesn't have much of a performance impact when it is disabled. Casual benchmark runs indicated the cost was less than the run-to-run variation.

The PR starts with a bunch of tidying up, including the patch from #271.

dcoutts commented 1 month ago

The main question is about how we want to treat the RocksDB benchmark. Is it worth keeping it in sync?

I think it is, yes. I think it's a useful reference point.

dcoutts commented 1 month ago

I'll do the syncing of the rocksdb benchmark as a follow-up.