search

atgambardella / pytorch-es

Evolution Strategies in PyTorch
MIT License
349 stars 37 forks source link

Performance on MountainCar #9

Closed wenyeming333 closed 7 years ago

wenyeming333 commented 7 years ago

Does anyone train the model on MountainCar-v0? I can only obtain the minimum reward -200. I tried both smaller sigma and larger sigma but none of both worked.

atgambardella commented 7 years ago

Hi, I also could not get the algorithm to work on MountainCar -- I would be interested if anyone could get it to work

atgambardella commented 7 years ago

With OpenAI's distributed ES, it appears that people have been able to get MountainCar working, but not ContinuousMountainCar: https://github.com/openai/evolution-strategies-starter/issues/9

atgambardella commented 7 years ago

For anyone interested: the main difference between OpenAI's code and mine seems to be the batch size. OpenAI collects a large number of samples (several thousand) before taking a gradient step, and this seems to be necessary for even mildly difficult problems.