Closed parkervg closed 4 months ago
Example: we can see that PR #18 by @zvs08 already has a definitive impact on runtime! Especially on the join-hungry '1966_nba_draft' task:
Task | Average Runtime | # Unique Queries |
---|---|---|
financials | 0.0467936 | 7 |
rugby | 0.267355 | 4 |
1966_nba_draft | 0.113532 | 2 |
Inspired by projects like duckdb, this PR adds a benchmark directory containing queries we can use for doing runtime tests.
You can invoke it via
python -m benchmark.run
. It will return a markdown table. For example, in this branch, we get the following results.The idea is that all future PRs will have the before/after benchmark results included.