search

data-apis / dataframe-api

RFC document, tooling and other content related to the dataframe API standard
https://data-apis.org/dataframe-api/draft/index.html
MIT License
102 stars 20 forks source link

Implementing TPC-H #271

Open amueller opened 1 year ago

amueller commented 1 year ago

Has anyone thought about implementing TPC-H using the dataframe API? I think this would be very useful to test the scope, and also to draw attention to the dataframe API. It would mean that anyone implementing the dataframe API could immediately get an apples-to-apples benchmark of performance.

Whether TPC-H is a good benchmark for dataframes is maybe not entirely clear, but it's the best there is right now AFAIK.

If we can make it so that polars, modin and duckdb run their comparisons via the dataframe API, I think that would be pretty sweet.

You can see the polars implementation of TPC-H here: https://github.com/pola-rs/tpch results here: https://www.pola.rs/benchmarks.html

MarcoGorelli commented 1 year ago

Great suggestion!

I'll try this out and see how far I get, it'll likely highlight some missing areas

It would require that the dataframe-api would have to be as close to a zero-cost abstraction as possible - https://github.com/data-apis/dataframe-api/pull/249 would bring us a lot closer to that goal, so if you had any input there I'd really appreciate it

thanks 🙏

amueller commented 1 year ago

Maybe should be tpc-ds or maybe tpcx-xBB

MarcoGorelli commented 1 year ago

We're added a couple of tpc-h examples here:

https://github.com/data-apis/dataframe-api/tree/main/spec/API_specification/examples/tpch

MarcoGorelli commented 2 weeks ago

Narwhals supports running all 22 TPC-H queries across supported backends (pandas, Polars, PyArrow, Dask, cuDF, Modin)