search

nomuramasahir0 / crfmnes

(CEC2022) Fast Moving Natural Evolution Strategy for High-Dimensional Problems
https://arxiv.org/abs/2201.11422
MIT License
18 stars 1 forks source link

Questions regarding the FM-NES/CR-FM-NES/DX-NES step size adaption #14

Closed millardjn closed 1 month ago

millardjn commented 2 months ago

Hello Masahiro,

Thank you for your work on these optimisers. I have implemented the CR-FM-NES algorithm in rust based on the C++ implementation linked on this repo.

While doing so I read through your papers, but did not find an explanation for why the algorithm does not adjust sigma using the older style Path-Length Control (cumulative step-size adaptation), and instead uses the the movement-stagnation-convergence search phase approach along with weighting of isotropic distance. It appears this was decided back in the paper "Proposal of distance-weighted exponential natural evolution strategies" which I do not have access too.

Does this adaption method produce better outcomes on most test problems? Does it mostly help with constrained problems? I was wondering if you could comment on how important this choice of step size adaption is and whether implementations should also expose to the user a choice of CSA step size adaption.

Thanks, James

nomuramasahir0 commented 1 month ago

Hi @millardjn ,

Thank you for your question. The step-size adaptation in FM-NES, CR-FM-NES, and DX-NES is based on the natural gradient update, as detailed in the xNES paper. However, which step-size adaptation method is most effective remains an open question (see, for example, this paper).

Does this adaption method produce better outcomes on most test problems? Does it mostly help with constrained problems?

We confirmed the effectiveness of this step-size adaptation on implicitly constrained problems in the DX-NES-IC paper. Please note that FM-NES is an extension of DX-NES-IC.

I was wondering if you could comment on how important this choice of step size adaption is and whether implementations should also expose to the user a choice of CSA step size adaption.

One of the key components of FM-NES, CR-FM-NES, and DX-NES is learning rate switching, which is grounded in the natural gradient perspective. Therefore, simply replacing the current step-size adaptation with other methods, such as CSA, may not be straightforward.

millardjn commented 1 month ago

Thank you very much for your answers, that helps greatly.