facebookresearch / nevergrad

A Python toolbox for performing gradient-free optimization
https://facebookresearch.github.io/nevergrad/
MIT License
3.92k stars 352 forks source link

Invalid state for DE unless num_ask - num_tell <= popsize #1461

Open Game4Move78 opened 2 years ago

Game4Move78 commented 2 years ago

Calling ask more than popsize times will result in an invalid state for DE. In particular it will cause the _uid_queue to wrap around, and produce two asked points with the same parent which adds the uid twice to _uid_queue on calls to tell. Then a tell-not-ask with better loss has the potential to kick the parent from the population, but leave its uid in the _uid_queue, resulting in a KeyError on one of the subsequent calls to ask.

If this is for performance reasons, the documentation could stress that calling ask no more than popsize times should be manually enforced as it can break some optimizers.

bottler commented 2 years ago

When the user creates an optimizer object and specifies num_workers, they are promising that they will never have more than the given number of untold asks. Nevergrad optimizers are not designed to handle this promise being broken.

Game4Move78 commented 2 years ago

Perhaps this could be mentioned in the docs as a promise rather than as typical usage.

https://github.com/facebookresearch/nevergrad/issues/263 is an example where a user other than myself thought they could ignore impossible-to-evaluate points without a tell.

The answer to https://github.com/facebookresearch/nevergrad/issues/1028 had given me the impression that it's not absolutely crucial to set popsize larger than num_workers, and that its just a good rule-of-thumb.

bottler commented 2 years ago

It seems like it would be good to add some more error checking for these cases.

Game4Move78 commented 2 years ago

This might be a safe way to do it. I don't know if this invariant matters for other optimizers.