Intel-bigdata / Spark-PMoF

Spark Shuffle Optimization with RDMA+AEP
Apache License 2.0
30 stars 22 forks source link

optimize pmem buffer in native code #63

Closed tanghaodong25 closed 4 years ago

tanghaodong25 commented 4 years ago

Current code needs to malloc new buffer, then copy content from existing buffer to new buffer when load data from persistent memory. The optimized code preserves buffer with wanted size when creating pmem inputream, then loads data from persistent memory in append mode.

Signed-off-by: Haodong Tang haodong.tang@intel.com