#428: allow swaps of empty clusters from target

[ppebay]

Fixes #428

[ppebay]

Key findings:

• the initial impression that the load-balancing under-performance of the clustering-based strategy was caused by the inability to swap with empty clusters was correct;
• what was incorrect however was the first assumption that it was caused by the non-existence of a notional empty cluster on the source rank; this is because the logic being to try to offload work from overloaded to underloaded ranks, there is no point in trying to swap and empty cluster on the source in exchange for something on the trial destination; instead the symmetric exchange should be tried.

As a result, the results of the call to __cluster_objects() on the destination rank must always be “enriched” with the notional empty cluster, denoted as: None: []

[ppebay]

Example of 16 iterations of the too-complex case, in non-deterministic mode:

N.B.: load imbalance continues to decrease and is not blocked at that level, past iteration 16.

[ppebay]

Verifying that has no negative impact on other, previously well resolved, cases:

[ppebay]

@nlslatt should I nonetheless merge, while you are further investigating?

[nlslatt]

@nlslatt should I nonetheless merge, while you are further investigating?

No, see Slack.

[ppebay]

Results with new execution logic (now only one target allowed to be picked but with an arbitrary numbers of swaps):

Much better results and with fewer iterations

[ppebay]

Further improved strategy, both for execution speed and load imbalance:

[ppebay]

Results also improved for the challenging_toy_hundreds_tasks case (also in non-deterministic mode):

[nlslatt]

@ppebay This is working better for me as well

[ppebay]

Thanks @nlslatt @lifflander , can we merge this PR?