Shmuma
commented
6 years ago Hi!
Yes, this is ok, as first iterations are used to populate replay buffer. Once it has enough samples, we start training, which slows down the process.
Closed garkavem closed 6 years ago
Shmuma
commented
6 years ago Hi!
Yes, this is ok, as first iterations are used to populate replay buffer. Once it has enough samples, we start training, which slows down the process.
Hi! I tried running the code and f/s dropped significantly after 10 episodes. Is it normal?
python3 05_new_wrappers.py WARN: <class ‘lib.atari_wrappers.FrameStack’> doesn’t implement ‘reset’ method, but it implements deprecated ‘_reset’ method. 935: done 1 games, mean reward -19.000, speed 281.93 f/s, eps 0.99 1694: done 2 games, mean reward -20.000, speed 334.75 f/s, eps 0.98 2502: done 3 games, mean reward -20.333, speed 338.90 f/s, eps 0.97 3441: done 4 games, mean reward -20.500, speed 335.34 f/s, eps 0.97 4537: done 5 games, mean reward -20.200, speed 289.07 f/s, eps 0.95 5531: done 6 games, mean reward -19.833, speed 330.80 f/s, eps 0.94 6649: done 7 games, mean reward -19.714, speed 335.40 f/s, eps 0.93 7648: done 8 games, mean reward -19.625, speed 334.24 f/s, eps 0.92 8427: done 9 games, mean reward -19.778, speed 331.16 f/s, eps 0.92 9462: done 10 games, mean reward -19.700, speed 333.20 f/s, eps 0.91 10399: done 11 games, mean reward -19.818, speed 40.25 f/s, eps 0.90 11157: done 12 games, mean reward -19.917, speed 18.19 f/s, eps 0.89 12234: done 13 games, mean reward -19.769, speed 17.30 f/s, eps 0.88 13305: done 14 games, mean reward -19.714, speed 16.67 f/s, eps 0.87 14345: done 15 games, mean reward -19.733, speed 16.37 f/s, eps 0.86 15368: done 16 games, mean reward -19.688, speed 16.10 f/s, eps 0.85 16308: done 17 games, mean reward -19.706, speed 15.96 f/s, eps 0.84 17303: done 18 games, mean reward -19.667, speed 15.72 f/s, eps 0.83 18406: done 19 games, mean reward -19.632, speed 15.95 f/s, eps 0.82 19307: done 20 games, mean reward -19.700, speed 15.08 f/s, eps 0.81 20146: done 21 games, mean reward -19.714, speed 16.20 f/s, eps 0.80 21251: done 22 games, mean reward -19.727, speed 16.02 f/s, eps 0.79 22008: done 23 games, mean reward -19.783, speed 15.60 f/s, eps 0.78 22968: done 24 games, mean reward -19.750, speed 15.50 f/s, eps 0.77 23731: done 25 games, mean reward -19.800, speed 16.23 f/s, eps 0.76 24857: done 26 games, mean reward -19.769, speed 16.67 f/s, eps 0.75 25617: done 27 games, mean reward -19.815, speed 16.48 f/s, eps 0.74 26535: done 28 games, mean reward -19.857, speed 16.76 f/s, eps 0.73 27413: done 29 games, mean reward -19.897, speed 16.02 f/s, eps 0.73 28251: done 30 games, mean reward -19.900, speed 16.86 f/s, eps 0.72 29279: done 31 games, mean reward -19.871, speed 15.92 f/s, eps 0.71