search

voltrondata-labs / arrowbench

R package for benchmarking
Other
13 stars 9 forks source link

Align results with those of labs/benchmarks and the Conbench API #116

Closed alistaire47 closed 1 year ago

alistaire47 commented 1 year ago

This PR changes a number of things, but all with the goal of aligning the JSON results arrowbench outputs with those currently posted to the Conbench API via voltrondata-labs/benchmarks so that eventually arrowbench can post its own results directly.

Changes fall into two categories:

  1. Changes to result structure to align with the /api/benchmarks/ endpoint
  2. Changes to benchmarks so the data that populates results aligns with what labs/benchmarks generates. This has only been done for the set of benchmarks run by arrow-benchmarks-ci.

This PR cannot be merged until https://github.com/voltrondata-labs/benchmarks/pull/122 is merged, or it will break current arrow benchmarks. That PR contains code to handle changes to assorted names and parameter names here. It has been tested with that branch.

alistaire47 commented 1 year ago

Failing build is a DuckDB thing that I believe will be addressed by https://github.com/voltrondata-labs/arrowbench/pull/113

alistaire47 commented 1 year ago

arrow-benchmarks-ci buildkite test build (together with voltrondata-labs/benchmarks#122) passes: https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-arm64-m6g-linux-compute/builds/1740#0184ca79-7c13-407e-ad17-814eed712a28/6-14249

jonkeane commented 1 year ago

Oh, and also that release check is actually fixed by https://github.com/voltrondata-labs/arrowbench/pull/117/files#diff-e35a10b8001b4a29a3a67979b010eebbf6285e9b0369c78693662e52d7b9e3b3R12 it's ultimately not going to be needed once we move to datalogistik, but it'll be fixed in main when #117 is merged