mlpack / ensmallen

A header-only C++ library for numerical optimization --
http://ensmallen.org
Other
742 stars 120 forks source link

IPOP and BIPOP restart mechanisms #403

Open beny-bee opened 4 months ago

beny-bee commented 4 months ago

Hi everyone!

I've taken a break from the termination criteria and have been implementing restart mechanisms for CMA-ES. The plan was to do IPOP, but adding BIPOP was quite simple so I suggest adding it too.

IPOP: is an extension of the standard CMA-ES that features an increasing population size strategy, where at each termination of the optimizer the population is increased by a constant factor. This is based on the initial work found in PR #373.

BIPOP: is pretty straightforward after setting up IPOP. BIPOP uses two interlaced strategies: one with an increase in population size with a constant factor (identical to IPOP) and another with a smaller, varying population size. I've seen some places that this strategy works better on certain functions.

Additionally, there are two things to consider:

rcurtin commented 3 months ago

Don't forget to add documentation to doc/. Without documentation nobody will ever know about these features.

beny-bee commented 3 months ago

Completely forgot about the documentation, added it now.

zoq commented 2 months ago

@mlpack-jenkins test this please

rcurtin commented 2 weeks ago

@beny-bee @zoq does anything else need to be done here? It looks to me like this one is pretty much ready to go, but I don't know the state of it. It would be great to get it merged. :+1:

beny-bee commented 2 weeks ago

@rcurtin should be ready to be merged!