Eventual-Inc / Daft

Distributed DataFrame for Python designed for the cloud, powered by Rust
https://getdaft.io
Apache License 2.0
1.79k stars 108 forks source link

Limit Pushdowns for Left/Right/Outer joins #2275

Open kevinzwang opened 2 weeks ago

kevinzwang commented 2 weeks ago

Is your feature request related to a problem? Please describe. One optimization we can make when we do a limit after a join is to push down the limit through the join when it is a left, right, or outer join. In those cases, we can bound the amount of rows we need on the left, right, or both side dataframes respectively to the limit.

Describe the solution you'd like Implementation of limit pushdowns to the relevant dataframes on left, right, and outer joins.

Describe alternatives you've considered N/A

Additional context N/A