search

openucx / sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer
https://www.sparkucx.org/
BSD 3-Clause "New" or "Revised" License
45 stars 31 forks source link

[CORE] Spark-3.0 reducer protocol. #21

Closed petro-rudenko closed 4 years ago

petro-rudenko commented 4 years ago

Spark-3.0 reducer protocol that handles ShuffleBlockBatchId blocks, that has type (mapId, startReduceId, endReduceId). Also handles translation from MapId -> BlockIdx, bacuse mapId now unique long, while we need integer offset.

sonarcloud[bot] commented 4 years ago

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication