apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
5.62k stars 1.05k forks source link

Support join filter in NestedLoopJoin in fizz join test cases #10787

Open edmondop opened 1 month ago

edmondop commented 1 month ago

Is your feature request related to a problem or challenge?

https://github.com/apache/datafusion/pull/10728 closes the first part of #10659, however combining the on_filter with the provided join_filter produces wrong results,

---- fuzz_cases::join_fuzz::test_inner_join_1k_filtered stdout ----
thread 'fuzz_cases::join_fuzz::test_inner_join_1k_filtered' panicked at datafusion\core\tests\fuzz_cases\join_fuzz.rs:387:17:
assertion `left == right` failed: NestedLoopJoinExec and HashJoinExec produced different results
  left: (3, "| 21 | 21 | 2078626761  | 1773284061  | 21 | 25 | 1533589294  | 685853846   |")
 right: (3, "| 0  | 75 | -1848255860 | 30924674    | 0  | 75 | 1261908710  | 1064557805  |")
stack backtrace:

Describe the solution you'd like

Add the right logic to the nested inner join loop filter

Describe alternatives you've considered

No response

Additional context

No response

comphead commented 1 month ago

Thats interesting, I'll try to add a test case for it.