feat: Support Spark-3.1 version.

NVIDIA / sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

https://www.sparkucx.org/

BSD 3-Clause "New" or "Revised" License

22 stars 14 forks source link

feat: Support Spark-3.1 version. #3

Open trietopsoft opened 3 years ago

trietopsoft commented 3 years ago

Adding Spark 3.1 function calls for reader/writer, specifically: startMapIndex: Int and endMapIndex: Int. Pull request follows existing support for 3.0, etc.

petro-rudenko commented 3 years ago

Can you pease also add spark-3.1 to ci script: https://github.com/NVIDIA/sparkucx/blob/master/.github/workflows/sparkucx-ci.yml#L12

trietopsoft commented 3 years ago

ci script updated