[FEAT] connect: `df.join`

[andrewgazelka]

• [ ] fix

  E           pyspark.errors.exceptions.connect.SparkConnectGrpcException: <_MultiThreadedRendezvous of RPC that terminated with:
  E               status = StatusCode.INTERNAL
  E               details = "Error in Daft server: Not Yet Implemented: Joins without join conditions (cross join) are not supported yet
  E           
  E           Location:
  E               src/daft-connect/src/op/execute/root.rs:37:39"
  E               debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:55241 {created_time:"2024-11-21T09:18:54.065079-08:00", grpc_status:13, grpc_message:"Error in Daft server: Not Yet Implemented: Joins without join conditions (cross join) are not supported yet\n\nLocation:\n    src/daft-connect/src/op/execute/root.rs:37:39"}"
  E           >

[andrewgazelka]

[!WARNING] This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite. Learn more

• #3354 👈 (View in Graphite)
• #3386
• #3373 : 5 other dependent PRs (#3375 , #3377 , #3379 and 2 others)
• #3359
• #3351
• #3350 : 3 other dependent PRs (#3346 , #3376 , #3387 )
• #3362 : 1 other dependent PR (#3363 )
• #3355 : 2 other dependent PRs (#3357 , #3360 )
• #3342 : 6 other dependent PRs (#3344 , #3345 , #3348 and 3 others)
• #3341 : 3 other dependent PRs (#3309 , #3326 , #3343 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[graphite-app[bot]]

Graphite Automations

"Notify author when CI fails" took an action on this PR • (11/21/24)

1 teammate was notified to this PR based on Andrew Gazelka's automation.

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #3354 will degrade performances by 54.24%

_{Comparing andrew/connect-join (2c4cb41) with main (cbe9d3b)}

Summary

❌ 2 regressions
✅ 15 untouched benchmarks

:warning: Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	andrew/connect-join	Change
❌	test_iter_rows_first_row[100 Small Files]	219.4 ms	262.8 ms	-16.51%
❌	test_show[100 Small Files]	15 ms	32.7 ms	-54.24%

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 83.01887% with 9 lines in your changes missing coverage. Please review.

Project coverage is 77.35%. Comparing base (3394a66) to head (a56ba92).

Files with missing lines	Patch %	Lines
.../daft-connect/src/translation/logical_plan/join.rs	82.69%	9 Missing :warning:

Additional details and impacted files

[](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc) ```diff @@ Coverage Diff @@ ## main #3354 +/- ## ======================================= Coverage 77.35% 77.35% ======================================= Files 685 686 +1 Lines 83631 83682 +51 ======================================= + Hits 64695 64735 +40 - Misses 18936 18947 +11 ``` | [Files with missing lines](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3354?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc) | Coverage Δ | | |---|---|---| | [src/daft-connect/src/translation/logical\_plan.rs](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3354?src=pr&el=tree&filepath=src%2Fdaft-connect%2Fsrc%2Ftranslation%2Flogical_plan.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc#diff-c3JjL2RhZnQtY29ubmVjdC9zcmMvdHJhbnNsYXRpb24vbG9naWNhbF9wbGFuLnJz) | `64.28% <100.00%> (+2.74%)` | :arrow_up: | | [.../daft-connect/src/translation/logical\_plan/join.rs](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3354?src=pr&el=tree&filepath=src%2Fdaft-connect%2Fsrc%2Ftranslation%2Flogical_plan%2Fjoin.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc#diff-c3JjL2RhZnQtY29ubmVjdC9zcmMvdHJhbnNsYXRpb24vbG9naWNhbF9wbGFuL2pvaW4ucnM=) | `82.69% <82.69%> (ø)` | | ... and [3 files with indirect coverage changes](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3354/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc)

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests