search

maurock / snake-ga

AI Agent that learns how to play Snake with Deep Q-Learning
531 stars 1k forks source link

The snake often get stuck in a loop during training #8

Closed cfmak closed 2 years ago

cfmak commented 4 years ago

The snake often get stuck in a loop during training, a.k.a. going in a circle, and wouldn't end some training game. This happens as early as Game 2, or Game 4 sometimes.

mrwang33 commented 4 years ago

also happened to me ~

carsondobiash commented 4 years ago

Happens for me as well!

LucasColas commented 4 years ago

And the snake stays in this position ? He doesn't move anymore ?

maurock commented 4 years ago

If the agent hasn't properly explored the state and action space, especially during the first phases of training, it would likely choose the same sequence of actions. If the epsilon parameter goes to zero too fast, the agent would favor a repeated sequence of actions. A solution is to tweak the epsilon parameter decay to guarantee proper exploration. To know more about this, please check the Exploration vs. Exploitation trade-off in Reinforcement Learning.