Closed ppebay closed 1 year ago
Randomizing traversal order typical results in a different behavior of the LB iterations (below with the Recursive
strategy) --- although we note than an optimal configuration is nonetheless attained:
In contrast, when deterministic_transfer
is set to True
, it is serendipitous but nonetheless observable that always starting with rank 0 and continuing in standard index order allows for faster convergence:
In the case of the Clustering
strategy, by adding another "layer" of randomization (in cluster selection as discussed @lifflander) we can indeed reproduce less-than-optimal results sometimes:
In contrast when deterministic_transfer
is set to True
we always find the known optimal value of 90.
@cwschilly I am leaving the PR in this state, when you get a chance can you please look into the remaining CI failures? Thanks
@ppebay I just had to rebase on develop
to get @tlamonthezie 's CI fix (#420) -- the PR now passes all tests
@lifflander @ppebay This PR is ready for review
Will let @cwschilly now test the CI as discussed with ordered sets for the targets
when transfer phase is deterministic
@ppebay @lifflander I have run the acceptance test on this PR several times and analyzed the output, comparing with other runs that have failed. This PR consistently passes the test. I suspect it has to do with changing deterministic_transfer
to True
in conf.yaml
. I believe this resolves #375
Resolves #414
Also improved the code as a result of the review of the control flow, by factoring out commonalities between the 2 transfer strategies and moving them to the abstract base strategy.
Resolves #375 as well, as discovered by @cwschilly