CMAbounded and CMApara: errors and warnings depending on number of workers and parameters; Nevergrad version 1.0.1

[nepfaff]

Steps to reproduce

The error is identical to the one pointed out here. The observations about the number of workers and the number of parameters hold for both issues. However, that issue is closed, so I'm opening a new issue for visibility.

from concurrent import futures

import nevergrad as ng
import numpy as np

x_init = ng.p.Array(shape=(1000,))
x_init.set_bounds(lower=-5, upper=5)

def norm(x):
    return np.linalg.norm(x)

# optimizer = ng.optimizers.CMAbounded(
#     parametrization=x_init, budget=5000, num_workers=10
# )
optimizer = ng.optimizers.CMApara(parametrization=x_init, budget=5000, num_workers=10)
with futures.ProcessPoolExecutor(max_workers=optimizer.num_workers) as executor:
    recommendation = optimizer.minimize(norm, executor=executor, batch_mode=False)

Observed Results

The exact same warnings as in the linked issue. My minimal example doesn't produce the runtime error. However, I'm getting the same runtime error in my actual codebase.

These issues don't occur if I use CMAstd.

Output of pip list

Note that Nevergrad has version 1.0.1.

ackage                  Version      Editable project location
------------------------ ------------ ----------------------------------
antlr4-python3-runtime   4.9.3
appdirs                  1.4.4
asttokens                2.4.0
attrs                    23.1.0
backcall                 0.2.0
bayesian-optimization    1.4.3
black                    23.9.1
build                    0.10.0
CacheControl             0.13.1
certifi                  2023.7.22
cffi                     1.15.1
cfgv                     3.4.0
charset-normalizer       3.2.0
cleo                     2.0.1
click                    8.1.7
cloudpickle              2.2.1
cma                      3.3.0
cmake                    3.27.7
colorama                 0.4.6
contourpy                1.1.1
crashtest                0.4.1
cryptography             41.0.4
cycler                   0.11.0
decorator                5.1.1
distlib                  0.3.7
docker-pycreds           0.4.0
drake                    1.24.0
dulwich                  0.21.6
exceptiongroup           1.1.3
executing                1.2.0
filelock                 3.12.4
fonttools                4.42.1
gitdb                    4.0.11
GitPython                3.1.40
gradescope-utils         0.5.0
hydra-core               1.3.2
identify                 2.5.29
idna                     3.4
importlib-metadata       6.8.0
installer                0.7.0
ipython                  8.15.0
isort                    5.12.0
jaraco.classes           3.3.0
jedi                     0.19.0
jeepney                  0.8.0
Jinja2                   3.1.2
joblib                   1.3.2
jsonschema               4.17.3
keyring                  24.2.0
kiwisolver               1.4.5
lit                      17.0.3
manipulation             2023.9.21
MarkupSafe               2.1.3
matplotlib               3.8.2
matplotlib-inline        0.1.6
more-itertools           10.1.0
mpld3                    0.5.9
mpmath                   1.3.0
msgpack                  1.0.5
mypy-extensions          1.0.0
networkx                 3.2.1
nevergrad                1.0.1
nlopt                    2.7.1
nodeenv                  1.8.0
numpy                    1.26.2
omegaconf                2.3.0
packaging                23.1
pandas                   2.1.4
parso                    0.8.3
pathspec                 0.11.2
pexpect                  4.8.0
pickleshare              0.7.5
Pillow                   10.0.1
pip                      23.3.1
pkginfo                  1.9.6
platformdirs             3.10.0
plotly                   5.18.0
poetry                   1.6.1
poetry-core              1.7.0
poetry-plugin-export     1.5.0
poetry-pre-commit-plugin 0.1.2
pre-commit               3.4.0
prompt-toolkit           3.0.39
protobuf                 4.25.1
psutil                   5.9.6
ptyprocess               0.7.0
pure-eval                0.2.2
pybind11                 2.11.1
pycparser                2.21
pydot                    1.4.2
Pygments                 2.16.1
pyparsing                3.1.1
pyproject_hooks          1.0.0
pyrsistent               0.19.3
python-dateutil          2.8.2
pytz                     2023.3.post1
PyYAML                   6.0.1
rapidfuzz                2.15.1
requests                 2.31.0
requests-toolbelt        1.0.0
scikit-learn             1.3.2
scipy                    1.11.4
SecretStorage            3.3.3
sentry-sdk               1.37.1
setproctitle             1.3.3
setuptools               68.2.2
shellingham              1.5.3
six                      1.16.0
smmap                    5.0.1
stack-data               0.6.2
sympy                    1.12
tenacity                 8.2.3
threadpoolctl            3.2.0
timeout-decorator        0.5.0
tomli                    2.0.1
tomlkit                  0.12.1
torch                    2.0.1+cu118
tqdm                     4.66.1
traitlets                5.10.0
triton                   2.0.0
trove-classifiers        2023.9.19
typing_extensions        4.8.0
tzdata                   2023.4
urllib3                  2.0.5
virtualenv               20.24.5
wandb                    0.16.0
wcwidth                  0.2.6
zipp                     3.17.0

[teytaud]

No crash and no warning with the current github/main. Thx @nepfaff .