nuno-faria
commented
4 years ago The most important thing you should start looking at is the state analysed by the agent. Right now, it only looks at four:
- Number of lines cleared
- Number of holes
- Bumpiness (sum of the difference between heights of adjacent pairs of columns)
- Total Height
Maybe more information (such as the board max/min height, the current piece, the next piece) could help the agent to converge more consistently and achieve a better score. However, as more variables are added, the more time it should take for the agent to learn.
You could also try and change the exploration variable (epsilon), to make the agent play more/less random games.
Finally, you could also try and change the underlying architecture, such as increasing the number of neurons and layers.
However, I found that after some learning the agent keeps going without stopping. So, improving the agent could mean it plays not only to infinity, but also smarter, i.e. always trying to clear multiple lines at a time. To do this, I recommend decreasing the number of max_steps (currently without limit), so you train the agent to get a better score with the least number of pieces instead of just surviving.
Good luck on your project!
Hello, I am an engineer student, as a school project in reinforcement learning, we tried to give a look at your code and see if there were any way to improve even more the agent. So far, we'v etried changing some parameters and implemented behavioral cloning but haven't had much result. I also tried to train the AI with different iteration number and noticed that it doesn't affect the result as much as I thought. For example, sometimes I had better results with 2000 episodes than 10000. Do you have any idea for us to work with ? Thanks you !