Run Gym.CartPole-v1

[ieow]

Hi, I am trying to run gpu-implementation es.py using gym enviroment gym.CartPole-v1 I edited the configuration file es_atari_config.json by changing "frostbrite" to "gym.CartPole-V1" and encounter error below

2018-08-18 12:09:54.669655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
08/18/2018 12:09:54 PM {
    "episode_cutoff_mode": 5000,
    "game": "gym.CartPole-v1",
    "l2coeff": 0.005,
    "model": "ModelVirtualBN",
    "mutation_power": 0.02,
    "num_test_episodes": 200,
    "num_validation_episodes": 30,
    "optimizer": {
        "args": {
            "stepsize": 0.01
        },
        "type": "adam"
    },
    "population_size": 5000,
    "return_proc_mode": "centered_rank",
    "timesteps": 250000000.0
}
08/18/2018 12:09:54 PM Logging to: /tmp/tmp59hdhjso
2018-08-18 12:09:54.745411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 12:09:54.745453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 12:09:54.745472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-08-18 12:09:54.745476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-08-18 12:09:54.745588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
Traceback (most recent call last):
  File "es.py", line 293, in <module>
    main(**exp)
  File "es.py", line 148, in main
    worker = ConcurrentWorkers(make_env, Model, batch_size=64)
  File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/neuroevolution/concurrent_worker.py", line 135, in __init__
    ref_batch = gym_tensorflow.get_ref_batch(make_env_f, sess, 128)
  File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/gym_tensorflow/__init__.py", line 19, in get_ref_batch
    assert env.discrete_action
AssertionError

I notice that the GymEnv in tf_env.py having property of discrete_action that will only return False which is triggering the assert in the get_ref_batch

class GymEnv(PythonEnv): ... @property def discrete_action(self): return False

Is openai gym's enviroment ready for run?

[VashishtMadhavan]

The code only supports discrete action environments currently. Please use the non-gpu version for continuous action spaces.

[ieow]

Based on my understanding, openai gym's CartPole-v0 is a discrete action environment. https://github.com/openai/gym/wiki/CartPole-v0

Please correct me if I am wrong.

[fps7806]

@ieow you are correct, the Gym wrapper I created was experimental and clearly has some issues. The appropriate way is probably to check if the gym environment return a spaces.Box or spaces.Discrete. Either way, gym.* environments still needs some work. For now, you can just modify it to return isinstance(env[0].action_space, gym.spaces.Discrete)

[ieow]

Thanks. Somehow, I managed to get it running after some modification based on tf_atari.py

[fps7806]

@ieow Glad to hear, I'll reopen the ticket until we fix it, feel free to make a pull request if you are comfortable with your solution.

[ieow]

Pull request submitted

15