apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.28k stars 1.19k forks source link

Expose TypeCoercionRewriter::new #13375

Open niebayes opened 1 day ago

niebayes commented 1 day ago

Is your feature request related to a problem or challenge?

Our database uses sqlparser to parse a custom SQL statement, and we skip the logical plan to directly construct an execution plan with DataFusion. This SQL statement includes a WHERE clause, so we need to construct a FilterExec. During the process of building FilterExec, we found that the create_physical_expr method from the datafusion-physical-expr module doesn’t perform type coercion, but instead assumes all expressions have already undergone type coercion.

To implement type coercion, we reviewed the DataFusion source code and found that from higher-level APIs to lower-level ones, there are several approaches that support type coercion for expressions:

In our case, we only want to perform type coercion on expressions, so TypeCoercionRewriter seems to be the best fit. However, we’re unable to create a TypeCoercionRewriter instance because its new method isn’t public.

We would like DataFusion to expose TypeCoercionRewriter::new. If there is another, simpler method to achieve this, please let us know. Thank you.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

niebayes commented 1 day ago

If this change is reasonable, I would like to file a PR.

Omega359 commented 21 hours ago

I did find this ticket where some of those changes landed which had some doc updates that may be helpful. I'll leave it to others more intimate in that area of the code to speak about your exact request.

niebayes commented 21 hours ago

@Omega359 Thank you!

jonahgao commented 17 hours ago

Since TypeCoercionRewriter itself and some of its methods are already public, this change makes sense to me.

niebayes commented 13 hours ago

@jonahgao @Omega359 Hi, I have filed a PR to expose TypeCoercionRewriter::new and add some examples demonstrating how to apply type coercion to expressions. Please review it. See: https://github.com/apache/datafusion/pull/13387