apache / incubator-uniffle

Uniffle is a high performance, general purpose Remote Shuffle Service.
https://uniffle.apache.org/
Apache License 2.0
387 stars 149 forks source link

[MINOR] feat(spark-client): Support set accessId by another config dynamically #2250

Closed maobaolong closed 4 days ago

maobaolong commented 6 days ago

What changes were proposed in this pull request?

Support set accessId by another config dynamically.

Why are the changes needed?

Without this PR, we have to set accessId for each user belongs to spark.yarn.queue, after patch this feature, the spark.access.id set to the value of spark.yarn.queue, if we set spark.rss.access.id.provider.key=spark.yarn.queue

Does this PR introduce any user-facing change?

Introduce spark new config spark.rss.access.id.provider.key.

How was this patch tested?

Updated UT.

codecov-commenter commented 6 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 52.88%. Comparing base (477bc30) to head (209ecee). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #2250 +/- ## ============================================ + Coverage 52.65% 52.88% +0.23% - Complexity 3376 3552 +176 ============================================ Files 514 532 +18 Lines 27565 29315 +1750 Branches 2582 2724 +142 ============================================ + Hits 14513 15502 +989 - Misses 12092 12824 +732 - Partials 960 989 +29 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 6 days ago

Test Results

 2 956 files  ±0   2 956 suites  ±0   6h 28m 2s ⏱️ + 17m 40s  1 092 tests ±0   1 090 ✅ ±0   2 💤 ±0  0 ❌ ±0  13 685 runs  ±0  13 655 ✅ ±0  30 💤 ±0  0 ❌ ±0 

Results for commit 923a4106. ± Comparison against base commit 4eebbd46.

:recycle: This comment has been updated with latest results.

jerqi commented 4 days ago

Could you add document, too?

maobaolong commented 4 days ago

@jerqi Thanks for remind me, document added, PTAL.

maobaolong commented 4 days ago

@jerqi Thanks for your review, merge it.