Fix bug: to_spark will fail if number of executors > 1 when block format is bytes

oap-project / raydp

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.

Apache License 2.0

293 stars 66 forks source link

Fix bug: to_spark will fail if number of executors > 1 when block format is bytes #349

Closed kira-lin closed 1 year ago

kira-lin commented 1 year ago

to_spark will fail if number of executors > 1 when block format is bytes, seems like if an executor get objects originally created by other executors, it will stuck.