[TPC-H] Move Dask queries to their own file

coiled / benchmarks

BSD 3-Clause "New" or "Revised" License

32 stars 17 forks source link

[TPC-H] Move Dask queries to their own file #1324

Closed hendrikmakait closed 9 months ago

hendrikmakait commented 9 months ago

This PR refactors the TPC-H tests so that we have functions that provide us with the queries in another file.

This is 99.9% verbatim copy+paste, I don't touch the queries themselves.

milesgranger commented 9 months ago

Is there backstory here, what is the motivation for this?

I only ask because in the early days of the TPCH implementation I made some maybe premature abstractions of queries to their own files somewhat similar to this (although again, I had each query in its own file so a bit different) and it was not well received and resulted in PRs like https://github.com/coiled/benchmarks/pull/1058/files which moved them into the test_<<enigne>>.py format we have now.

hendrikmakait commented 9 months ago

There are two major use cases for this:

Generally, I'd like to be able to generate (and run) a TPC-H query without the test harness
I want to add a benchmark for the plan generation and optimization in dask-expr for TPC-H queries.

hendrikmakait commented 9 months ago

This is a step toward #1325