Closed pdet closed 6 months ago
Do you need first to handle queries 11 and 15? Substrait defines any_value which is essentially the same except it doesn't require the input data to be ordered.
any_value
Thanks for pointing it out; indeed, I think any_value
should be sufficient; I will add the transformation!
This PR extends the usage of the Substrait Extension YAML files, so we internally take aggregation functions into consideration.
We previously disabled the 3001 warning checks from the substrate validator (failed to resolve anchor: URI anchor 0 does not exist). Now, we fail the tests if they occur. The validator is only executed in the TPC-H queries, and for TPC-H, we are currently missing the following functions (i.e., they are not defined in the substrate extension YAML files):
So, as a consequence of this, queries 11, 13, and 15 will now fail our substrait-validation tests.
Additionally, this PR also introduces a strict mode. When converting a query to a substrate plan, certain warnings/errors can be collected, and the conversion will fail instead of attempting a best-effort transformation. By default, this is set to false, but it can be used as follows: