search

NVIDIA / sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer
https://www.sparkucx.org/
BSD 3-Clause "New" or "Revised" License
22 stars 14 forks source link

Fix invalid capacity when `getByteBufferView` of 2G block #32

Open jeynmann opened 10 months ago

jeynmann commented 10 months ago

Why? capacity of DirectBuffer is int type. getByteBufferView of a 2G size block will result in overflow.

How? Use real message buffer size instead of block size.

petro-rudenko commented 10 months ago

Can you please rebase on master - i've fixed CI to run checks