apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.26k stars 1.18k forks source link

Eliminate more outer joins by supporting more expressions #13232

Open Dandandan opened 5 days ago

Dandandan commented 5 days ago

Is your feature request related to a problem or challenge?

This PR by @JasonLi-cn tried to support more pushdown of filters https://github.com/apache/datafusion/pull/13184 , however the implementation wasn't 100% right as it pushed down more predicates, but didn't change the join type (left to inner).

Describe the solution you'd like

Add support for a wider range of expressions in EliminateOuterJoin, notably in extract_non_nullable_columns. We lack support of more complex nested expressions, quite some binary operators, UDFs etc. such as:

We should aim to support any operator that returns null when the input value is null (most binary / unary operators).

Describe alternatives you've considered

No response

Additional context

No response

Rachelint commented 4 days ago

take

JasonLi-cn commented 4 days ago

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: https://github.com/apache/datafusion/pull/13249

Rachelint commented 4 days ago

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: #13249

😆 OK, I am learning it!