There is a fixed pattern in the "Pong" version

[Zeta36]

As long as in the file https://github.com/asrivat1/DeepLearningVideoGames/blob/master/Wrapped%20Game%20Code/pong_fun.py#L82, you have:

if self.bar2_y < self.circle_y + 7.5: self.bar2_y += ai_speed if self.bar2_y > self.circle_y - 42.5: #Here!! self.bar2_y -= ai_speed

That means that the classical IA will always fail in the same upper corner of the screen, and as you always start the game in the same velocity direction (7.,7.) after every game, there is a easy pattern that uses the NN to train and beat in the game always in the same way.

If you correct this line, for example, and change pong_fu.py to:

if self.bar2_y < self.circle_y + 7.5: self.bar2_y += ai_speed if self.bar2_y > self.circle_y - 7.5: #For this self.bar2_y -= ai_speed

The classical IA will not fail so much, and you can check that the best pong saved netowork (pong-dqn-1380000) is not so good playing as you can imagine. That's because the NN learned to find that error pattern in bar2, instead of learning to play any real game.

[WhiteHatArpit]

@Zeta36 If you would make such changes in the game, I suppose you would have to train the network again from the start.