search

spiraldb / vortex

A toolkit for working with compressed Arrow in-memory, on-disk, and over-the-wire
Apache License 2.0
92 stars 5 forks source link

Consolidate on Criteria or Divan for benchmarking #344

Open gatesn opened 3 weeks ago

AdamGS commented 1 week ago

My 2c here - I wished I could fork Criterion multiple times, and it seems like Divan is actually maintained which is a big plus in my book

robert3005 commented 1 week ago

I agree with your point. Seeing criterion have 23 open prs is not a good look for a project with a well defined functionality

gatesn commented 5 days ago

Only issue I see atm is that Divan doesn't support async benchmarks? Any ideas how to workaround @AdamGS ?

AdamGS commented 4 days ago

The first thing I would try is to create the runtime as part of the setup, and then call Runtime::block_on or it's equivalent in the actual benchmark code. From reading the docs seems like there are some threading-related configs you might have to use, and hopefully everything will play nice together.

nvzqz commented 3 days ago

Hi! I plan to prioritize async benchmarks after I ship JSON output. 🙂 Follow https://github.com/nvzqz/divan/issues/39 for updates

From reading the docs seems like there are some threading-related configs you might have to use, and hopefully everything will play nice together.

That's separate and unrelated from async. Multi-threaded benchmarks are more so for measuring contention on locks and atomics.