Avoid using `synchronize_labels` for `combine` function

modin-project / modin

Modin: Scale your Pandas workflows by changing a single line of code

http://modin.readthedocs.io

Apache License 2.0

9.8k stars 651 forks source link

Avoid using `synchronize_labels` for `combine` function #7299

Closed anmyachev closed 3 months ago

anmyachev commented 3 months ago

Index assignment is a very fast operation, the main time is spent on serialization/deserialization. The idea is to combine index assignment with partition combination operation in one remote kernel.