Eliminate more outer joins by supporting more expressions

Dandandan commented 4 weeks ago

Is your feature request related to a problem or challenge?

This PR by @JasonLi-cn tried to support more pushdown of filters https://github.com/apache/datafusion/pull/13184 , however the implementation wasn't 100% right as it pushed down more predicates, but didn't change the join type (left to inner).

Describe the solution you'd like

Add support for a wider range of expressions in EliminateOuterJoin, notably in extract_non_nullable_columns. We lack support of more complex nested expressions, quite some binary operators, UDFs etc. such as:

abs(x) > 1
a + a > c + b

We should aim to support any operator that returns null when the input value is null (most binary / unary operators).

Describe alternatives you've considered

No response

Additional context

No response

Rachelint commented 3 weeks ago

take

JasonLi-cn commented 3 weeks ago

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: https://github.com/apache/datafusion/pull/13249

Rachelint commented 3 weeks ago

Rachelint

I have implemented a preliminary version and am happy to solve this Issue together. 😄

pr: #13249

😆 OK, I am learning it!

apache / datafusion