benchmark(python): Shift polars TPC-H benchmarks to main repo and include in CI

chitralverma commented 1 year ago

This is in reference to a discussion @ritchie46 and I were having on discord regarding the TPC-H benchmarks.

The idea is the following,

Implement all 22 TPC-H queries for Polars in the Polars TPC-H repo
Move these 22 polars queries alongwith the executor into a benchmark module in this repo
Modify CI to include benchmarks via github actions. For this we have following options
- run on release
- run on merge to main/ master branch
- run on PRs

this will clearly show the rate of improvement of the project in terms of performance.

armgilles commented 1 year ago

Maybe an idea about monitoring performance for polars pytest-codspeed. It's used by pydantic-core and seems pretty easy and robust to use to monitor a bump or regression in terms of performance in CI (exemple https://github.com/pydantic/pydantic-core/pull/337)

chitralverma commented 1 year ago

datafusion did something similar i believe

pola-rs / polars