timsaucer commented 1 month ago

Which issue does this PR close?

Closes #696.

Rationale for this change

This PR sets up a work flow to generate TPH-C 1Gb data set in CI, runs the 22 examples, and compares their results to the known answer file. By adding this PR we improve the robustness of our test suite.

What changes are included in this PR?

This PR adds the following changes:

Adds CI workflow to generate TPH-C data set and answer file
Adds pytest file to run all 22 examples and compare them to the answer file
Exposes substring operation that was missing. Sometime between 36.0.0 and 38.0.0 substr was performing the operation instead.

Are there any user-facing changes?

substring function is exposed in python.

timsaucer commented 1 month ago

It looks like CI is running correctly and also caching the data. I’ll rebase in the morning and get the PR ready to merge.

timsaucer commented 1 month ago

Closing in favor of https://github.com/apache/datafusion-python/pull/711

apache / datafusion-python

Tsaucer/prepare tpch examples for ci #710

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?