search

jenkspt / CMA-ES

A short implementation and demonstration of the Covariance Matrix Adaptation algorithm in numpy
10 stars 2 forks source link

Are there lack of the covariance matrix update process in cmaes.py? #1

Closed CodingCatMountain closed 1 year ago

CodingCatMountain commented 1 year ago

Hi, @jenkspt. Thank you for the sharing about cma-es algorithm! I wanna to ask about a question about the details in the cmaes.py. In this script, I think the key process is line 58-73. However, I can't find the the covariance matrix update process (The Equation 45-47 in https://arxiv.org/pdf/1604.00772.pdf) in there except for the mean update process (The Equation 4 in https://arxiv.org/pdf/1604.00772.pdf) which is X = topk.mean(1,keepdims=True) + (E @ np.diag(np.sqrt(w)) @ N). I have to say CMA-ES algorithm is very complicate, and I am a green hand about it. So If I make some mistakes, please point them out! Thank you. Any discussions and instructions will be appreciated!

jenkspt commented 1 year ago

Hey @CodingCatMountain I'm glad you found this useful. I believe you are referring to the step size control? It's been awhile since I've looked at this - but I think this is a simplified version of CMA-ES as mentioned in section 4.

The covariance matrix adaptation, discussed in the last section, does not explicitly control the “overall scale” of the distribution, the step-size

The CMA-ES code definitely isn't simple - but is more complete. Maybe take a look at https://github.com/CMA-ES/pycma/blob/development/cma/purecma.py

CodingCatMountain commented 1 year ago

Thank you for replying! @jenkspt . Yeah, what I am referring to includes the step size control. Besides, I also wanna to mention the Rank-\mu-update or Rank-1-update. :) But it may not be a big deal now cause I should just utilize the pycma to figure out my problem rather than try to build a wheel again. Thank you again, jenkspt.

jenkspt commented 1 year ago

Sorry I can't be more helpful - maybe at some point I'll revisit CMA-ES and make a better reproduction (possibly w/ Jax 😜).