CMandAS3 errors and warnings depending on number of workers and parameters

[nhansendev]

Steps to reproduce

Versions: Python 3.10.7 nevergrad 0.6.0

Using the code block below the following combinations produce either normal results, InjectionWarnings, or RuntimeErrors:

• Normal results: ~200 parameters or fewer, any number of workers
• InjectionWarnings: greater than 200 parameters and two or more workers
• RuntimeErrors: greater than 200 parameters and exactly one worker

Replacing the return value of the _loss function with zero or another constant prevents the issues from appearing, though this is not a particularly useful method.

The messages accompanying the warnings and errors both mention the ask function being called out-of-order.

Observed Results

InjectionWarnings:

/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 33, 'index': 1, 'counter': 3}
                        This could be a bug in the calling order/logics or due to
                        a too small popsize used in `ask()` or when only using
                        `ask(1)` repeatedly. Please check carefully.
                        In case this is desired, the warning can be surpressed with
                        ``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``

  warnings.warn("""orphanated injected solution %s
/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 35, 'index': 0, 'counter': 2}
                        This could be a bug in the calling order/logics or due to
                        a too small popsize used in `ask()` or when only using
                        `ask(1)` repeatedly. Please check carefully.
                        In case this is desired, the warning can be surpressed with
                        ``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``

  warnings.warn("""orphanated injected solution %s
/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 35, 'index': 1, 'counter': 3}
                        This could be a bug in the calling order/logics or due to
                        a too small popsize used in `ask()` or when only using
                        `ask(1)` repeatedly. Please check carefully.
                        In case this is desired, the warning can be surpressed with
                        ``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``

  warnings.warn("""orphanated injected solution %s

RuntimeError:

Traceback (most recent call last):
  File "/home/nate/Desktop/Atom/testing.py", line 840, in <module>
    main()
  File "/home/nate/Desktop/Atom/testing.py", line 832, in main
    optimizer.minimize(_loss)
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 694, in minimize
    args = self.ask()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
    candidate = self._internal_ask_candidate()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 1660, in _internal_ask_candidate
    candidate = opt.ask()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
    candidate = self._internal_ask_candidate()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 2279, in _internal_ask_candidate
    return opt.ask()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
    candidate = self._internal_ask_candidate()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 737, in _internal_ask_candidate
    return self.optim.ask()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
    candidate = self._internal_ask_candidate()
  File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 580, in _internal_ask_candidate
    self._to_be_asked.extend(self.es.ask())
  File "/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py", line 1922, in ask
    pop_geno = self.ask_geno(number, xmean, sigma_fac)
  File "/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py", line 2165, in ask_geno
    raise RuntimeError(
RuntimeError: Mean shift samples are expected but missing.
This happens if, for example, `ask` is called  more than once, without calling `tell`
(because the first call removes the samples from the injection list).
`cma.sigma_adaptation.CMAAdaptSigmaTPA` step-size adaptation generates mean shift
samples and relies on them. 
Using ``ask(1)`` for any subsequent calls of `ask` works OK and TPA works if the
first two samples from the first call are retained as first samples when calling `tell`. 
EXAMPLE: 
    X = es.ask()
    X.append(es.ask(1)[0])
    ...
    es.tell(X, ...)

Expected Results

It runs without errors/warnings

Relevant Code

import nevergrad as ng
import random

def _loss(*args, **kwargs):
    return random.random()

def main():
    params = 300
    workers = 2

    parametrization = ng.p.Array(shape=(params,), lower=-3, upper=3)

    optimizer = ng.optimizers.CMandAS3(
        parametrization=parametrization, budget=1000, num_workers=workers
    )

    optimizer.minimize(_loss)

if __name__ == "__main__":
    main()

[teytaud]

Thanks for pointing this out. I've run the code without error on the current Nevergrad. I'll double check with other configurations in case a Pypi package number might have an impact.

[teytaud]

In case someone reproduces the error with nevergrad >= 0.8.0, could you << conv env export >> so that I can see which package versions are responsible ? Thanks for pointing the issue, there were related bugs some time ago, I hope the current version fixes all bugs.

[teytaud]

Everything runs smoothly at least in my environment and after the recent fixes. Thank for your work @Obliman and others who commented related issues, this helps a lot.