search

facebookresearch / ReAgent

A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)
https://reagent.ai
BSD 3-Clause "New" or "Revised" License
3.56k stars 516 forks source link

Failing to run discrete_dqn_maxq_asteroids_v0 #134

Closed pshevche closed 5 years ago

pshevche commented 5 years ago

Used commit: 2901f36ce8b21225d541f1110a2c100861f59431

When running ml/rl/test/gym/run_gym.py -p=ml/rl/test/gym/discrete_dqn_maxq_asteroids_v0.json (as well as other Atari experiments), I observe the following error:

INFO:main:Running gym with params INFO:main:{'env': 'Asteroids-v0', 'model_type': 'pytorch_discrete_dqn', 'max_replay_memory_size': 100000, 'rl': {'gamma': 0.99, 'target_update_rate': 0.2, 'maxq_learning': 1, 'epsilon': 0.2, 'temperature': 0.35, 'softmax_policy': 0}, 'rainbow': {'double_q_learning': False, 'dueling_architecture': False}, 'training': {'layers': [-1, 128, 64, -1], 'activations': ['relu', 'relu', 'linear'], 'minibatch_size': 64, 'learning_rate': 0.001, 'optimizer': 'ADAM', 'lr_decay': 0.999, 'cnn_parameters': {'conv_dims': [3, 32, 16], 'conv_height_kernels': [8, 4], 'conv_width_kernels': [8, 4], 'pool_kernels_strides': [2, 2], 'pool_types': ['max', 'max']}}, 'run_details': {'num_episodes': 5001, 'max_steps': 200, 'train_every_ts': 1, 'train_after_ts': 1, 'test_every_ts': 2000, 'test_after_ts': 1, 'num_train_batches': 1, 'avg_over_num_episodes': 100}} Traceback (most recent call last): File "/home/pshevche/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/ptvsd_launcher.py", line 43, in main(ptvsdArgs) File "/home/pshevche/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/lib/python/ptvsd/main.py", line 410, in main run() File "/home/pshevche/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/lib/python/ptvsd/main.py", line 291, in run_file runpy.run_path(target, run_name='main') File "/home/pshevche/miniconda3/envs/horizon-dev/lib/python3.6/runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "/home/pshevche/miniconda3/envs/horizon-dev/lib/python3.6/runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "/home/pshevche/miniconda3/envs/horizon-dev/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/pshevche/Documents/ovgu/master_thesis/develop/horizon/Horizon/ml/rl/test/gym/run_gym.py", line 941, in main(args[1:]) File "/home/pshevche/Documents/ovgu/master_thesis/develop/horizon/Horizon/ml/rl/test/gym/run_gym.py", line 597, in main args.path_to_pickled_transitions, File "/home/pshevche/Documents/ovgu/master_thesis/develop/horizon/Horizon/ml/rl/test/gym/run_gym.py", line 657, in run_gym minimum_epsilon, File "/home/pshevche/Documents/ovgu/master_thesis/develop/horizon/Horizon/ml/rl/test/gym/open_ai_gym_environment.py", line 72, in init for a in range(self.action_dim)] File "/home/pshevche/Documents/ovgu/master_thesis/develop/horizon/Horizon/ml/rl/test/gym/open_ai_gym_environment.py", line 72, in for a in range(self.action_dim)] AttributeError: 'OpenAIGymEnvironment' object has no attribute 'state_dim'

Could you help me fix this issue? Maybe it was already fixed in some other commit? So far I know that the problem originates in open_ai_gym_environment.py, lines 114 - 121. Since the observation space for Atari environments is 3-dimensional, the state_dim attribute is never set, but is used in open_ai_gym_environment.py, line 72 anyways, because Atari environments have discrete actions.

Any help is highly appreciated, thank you in advance!

czxttkl commented 5 years ago

At this moment, we kind of lost the ability to support image-based environments because we are focusing on production applications which are usually not in image formats. We definitely want to recover such support but need some time.

pshevche commented 5 years ago

Thanks for a quick response. I'll close the issue for now.