search

germain-hug / Deep-RL-Keras

Keras Implementation of popular Deep RL Algorithms (A3C, DDQN, DDPG, Dueling DDQN)
533 stars 149 forks source link

Trouble with A3C and Breakout #7

Closed ChristophePRAT closed 6 years ago

ChristophePRAT commented 6 years ago

Hi, I was trying to use a3c with the game 'breakout', but some error popped:

my command : python3 main.py --type A3C --env BreakoutNoFrameskip-v4 --is_atari --nb_episodes 10000 --n_threads 4 the result:

Score: 0%| | 0/10000 [00:00<?, ? episodes/s]Exception in thread Thread-3: Traceback (most recent call last): File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread a = agent.policy_action(np.expand_dims(old_state, axis=0)) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0] File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict return self.model.predict(self.reshape(inp)) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict check_batch_axis=False) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)

Exception in thread Thread-4: Traceback (most recent call last): File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread a = agent.policy_action(np.expand_dims(old_state, axis=0)) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0] File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict return self.model.predict(self.reshape(inp)) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict check_batch_axis=False) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)

Exception in thread Thread-5: Traceback (most recent call last): File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread a = agent.policy_action(np.expand_dims(old_state, axis=0)) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0] File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict return self.model.predict(self.reshape(inp)) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict check_batch_axis=False) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)

Exception in thread Thread-6: Traceback (most recent call last): File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread a = agent.policy_action(np.expand_dims(old_state, axis=0)) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0] File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict return self.model.predict(self.reshape(inp)) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict check_batch_axis=False) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)

Traceback (most recent call last): File "main.py", line 115, in main() File "main.py", line 108, in main a = algo.policy_action(old_state) File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0] File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict return self.model.predict(self.reshape(inp)) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict check_batch_axis=False) File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)

Thank you!

germain-hug commented 6 years ago

Hi, thanks for reporting the error, There was indeed an error on the input shape when using Atari games on A3C, due to the number of consecutive frames. Just updated the code, it should work better now!