[CORE] Spark-3.0 mapper writer plugin implementation.

openucx / sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

https://www.sparkucx.org/

BSD 3-Clause "New" or "Revised" License

45 stars 31 forks source link

[CORE] Spark-3.0 mapper writer plugin implementation. #20

Closed petro-rudenko closed 4 years ago

petro-rudenko commented 4 years ago

Spark-3.0 mapper implementation through a new ShufflePluginIO. Basically UcxLocalDiskShuffleExecutorComponents uses all the same code as Spark's one with only changing blockResolver = ucxShuffleManager.shuffleBlockResolver

sonarcloud[bot] commented 4 years ago

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
No Duplication information