zarr-developers / perfcapture

Capture the performance of a computer system whilst running a set of benchmark workloads.
MIT License
2 stars 2 forks source link

Consider defining workloads & benchmarks in yaml to be consistent with `zarr_implementations` #12

Open JackKelly opened 1 year ago

JackKelly commented 1 year ago

In today's meeting for "Zarr Performance & Benchmarking (Europe-friendly time)", @JoshMoore described the Zarr Implementations project: The Zarr Implementations project collects "data in zarr / n5 format written by different implementations" and tests for compatibility. Zarr Implementations is related to - although distinct from - benchmarking. Specifically: it might be nice to benchmark Zarr Implementations.

With perfcapture's current API, it should be possible to call Zarr Implementations from perfcapture.Workload.run (probably using one perfcapture.Workload class per Zarr implementation).

But it might also be nice to harmonize the API, such that both perfcapture and Zarr Implementations use the same yaml structures to define the workloads.

Related: