search

spiceai / spiceai

A unified SQL query interface and portable runtime to locally materialize, accelerate, and query datasets from any database, data warehouse, or data lake.
https://docs.spiceai.org
Apache License 2.0
1.6k stars 66 forks source link

Implement IntervalMonthDayNano scalar conversion in DataFusion Unparser #1580

Closed phillipleblanc closed 2 days ago

phillipleblanc commented 3 weeks ago

Describe the bug The unparser code in DataFusion (responsible for turning LogicalPlans back into SQL statements) does not handle IntervalMonthDayNano scalar values, which causes several of the TPC-H queries to fail with the following error: Error: "query tpch_q4 to results: External error: This feature is not implemented: Unsupported scalar: IntervalMonthDayNano(\"237684487542793012780631851008\")"

The relevant code that needs to be implemented: https://github.com/apache/datafusion/blob/1db3263497532fda5167386781755463f53c00a4/datafusion/sql/src/unparser/expr.rs#L802

This affects the following benchmark queries:

phillipleblanc commented 3 weeks ago

Filed https://github.com/apache/datafusion/issues/10795 to track on the DataFusion side.

y-f-u commented 1 week ago

The upstream fix is done, will create a new spiceai fork branch and merge relative stuff in.

y-f-u commented 2 days ago

The upstream fix is merged, however, the issue is still not fixed for affected bench tests, as the str unparsed from this is interval '1 mons' which is not supported by dremio connector.

y-f-u commented 2 days ago

Closed this to open a more specific one for failing bench in dremio.