search

yangkevin2 / neurips2021-lap3

17 stars 0 forks source link

Question about how to use cmaes for Minigrid discrete problems #4

Closed gloaming2dawn closed 2 years ago

gloaming2dawn commented 2 years ago

Hi, I am confused how the CMA-ES method is used for Minigrid discrete problems, as the method is designed for continuous problems

yangkevin2 commented 2 years ago

Hi, it's true that CMA-ES is designed for continuous problems, so we optimize a continuous space and then discretize it. In Minigrid in particular, how this works is say for example you have a length-2 trajectory where at each step you can go up/down/left/right, then you would optimize an 8-dimensional vector V (length 2 x 4 choices at each step). Whichever of the first 4 entries of V is largest corresponds to which action you take at the first step, then whichever of the last 4 entries of V is largest corresponds to the action at the second step.

I think the code for Minigrid isn't in the repo yet, let me bump my coauthor on that again.