devsisters / DQN-tensorflow

Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning
MIT License
2.46k stars 765 forks source link

AssertionError: Cannot call env.step() before calling reset() #63

Open frenzytejask98 opened 5 years ago

frenzytejask98 commented 5 years ago

I don't know why this error shows up. Please help me out here.

[] GPU : 1.0000 2019-05-23 23:34:52.437286: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA {'_save_step': 500000, '_test_step': 50000, 'action_repeat': 4, 'backend': 'tf', 'batch_size': 32, 'cnn_format': 'NHWC', 'discount': 0.99, 'display': True, 'double_q': False, 'dueling': False, 'env_name': 'Breakout-v0', 'env_type': 'detail', 'ep_end': 0.1, 'ep_end_t': 100000, 'ep_start': 1.0, 'history_length': 4, 'learn_start': 50000.0, 'learning_rate': 0.00025, 'learning_rate_decay': 0.96, 'learning_rate_decay_step': 50000, 'learning_rate_minimum': 0.00025, 'max_delta': 1, 'max_reward': 1.0, 'max_step': 50000000, 'memory_size': 100000, 'min_delta': -1, 'min_reward': -1.0, 'model': 'm1', 'random_start': 30, 'scale': 10000, 'screen_height': 84, 'screen_width': 84, 'target_q_update_step': 10000, 'train_frequency': 4} WARNING:tensorflow:From /home/tejask98/Desktop/DQN-tensorflow/dqn/agent.py:225: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py:189: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use tf.global_variables_initializer instead. [] Loading checkpoints... [!] Load FAILED: checkpoints/Breakout-v0/min_delta--1/max_delta-1/history_length-4/train_frequency-4/target_q_update_step-10000/double_q-False/memory_size-100000/action_repeat-4/ep_end_t-100000/dueling-False/min_reward--1.0/backend-tf/random_start-30/scale-10000/env_type-detail/learning_rate_decay_step-50000/ep_start-1.0/screen_width-84/learn_start-50000.0/cnn_format-NHWC/learning_rate-0.00025/batch_size-32/discount-0.99/max_step-50000000/max_reward-1.0/learning_rate_decay-0.96/learning_rate_minimum-0.00025/env_name-Breakout-v0/ep_end-0.1/model-m1/screen_height-84/ Traceback (most recent call last): File "main.py", line 70, in tf.app.run() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "main.py", line 65, in main agent.train() File "/home/tejask98/Desktop/DQN-tensorflow/dqn/agent.py", line 43, in train screen, reward, action, terminal = self.env.new_random_game() File "/home/tejask98/Desktop/DQN-tensorflow/dqn/environment.py", line 28, in new_random_game self.new_game(True) File "/home/tejask98/Desktop/DQN-tensorflow/dqn/environment.py", line 23, in new_game self._step(0) File "/home/tejask98/Desktop/DQN-tensorflow/dqn/environment.py", line 35, in _step self.screen, self.reward, self.terminal, = self.env.step(action) File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 30, in step assert self._episode_started_at is not None, "Cannot call env.step() before calling reset()" AssertionError: Cannot call env.step() before calling reset()

KingCoCos commented 5 years ago

I think this might work, change self._screen = None to self._screen = self.env.reset() in def init in the environment.py file