Running Atari Ram Environments with PPO2

[Ashboy64]

I'm trying to run PPO2 on the 'Breakout-ram-v0' environment, but am getting a few errors. The command I am using to run the program is:

python -m baselines.run --alg=ppo2 --env=Breakout-ramNoFrameskip-v0 --num_timesteps=1e7 --network=mlp

Which gives the following error:

Process Process-3:
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 17, in worker
    ob = env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 318, in reset
    return self.env.reset()
Process Process-2:
Process Process-1:
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 309, in reset
    return self.observation(observation)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/atari_wrappers.py", line 147, in observation
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/color.hpp:255: error: (-2:Unspecified error) in function 'cv::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::Set<3, 4>; VDcn = cv::Set<1>; VDepth = cv::Set<0, 2, 5>; cv::SizePolicy sizePolicy = (cv::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 17, in worker
    ob = env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 318, in reset
    return self.env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 309, in reset
    return self.observation(observation)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/atari_wrappers.py", line 147, in observation
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/color.hpp:255: error: (-2:Unspecified error) in function 'cv::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::Set<3, 4>; VDcn = cv::Set<1>; VDepth = cv::Set<0, 2, 5>; cv::SizePolicy sizePolicy = (cv::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
Process Process-4:
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 17, in worker
    ob = env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 318, in reset
    return self.env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 309, in reset
    return self.observation(observation)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/atari_wrappers.py", line 147, in observation
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/color.hpp:255: error: (-2:Unspecified error) in function 'cv::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::Set<3, 4>; VDcn = cv::Set<1>; VDepth = cv::Set<0, 2, 5>; cv::SizePolicy sizePolicy = (cv::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 17, in worker
    ob = env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 318, in reset
    return self.env.reset()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/site-packages/gym/core.py", line 309, in reset
    return self.observation(observation)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/atari_wrappers.py", line 147, in observation
    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/color.hpp:255: error: (-2:Unspecified error) in function 'cv::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::Set<3, 4>; VDcn = cv::Set<1>; VDepth = cv::Set<0, 2, 5>; cv::SizePolicy sizePolicy = (cv::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/run.py", line 225, in <module>
    main()
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/run.py", line 199, in main
    model, env = train(args, extra_args)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/run.py", line 81, in train
    **alg_kwargs
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/ppo2/ppo2.py", line 113, in learn
    runner = Runner(env=env, model=model, nsteps=nsteps, gamma=gamma, lam=lam)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/ppo2/runner.py", line 14, in __init__
    super().__init__(env=env, model=model, nsteps=nsteps)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/runners.py", line 11, in __init__
    self.obs[:] = env.reset()
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/vec_frame_stack.py", line 27, in reset
    obs = self.venv.reset()
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 79, in reset
    return np.stack([remote.recv() for remote in self.remotes])
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 79, in <listcomp>
    return np.stack([remote.recv() for remote in self.remotes])
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError 

If I use this command instead:

python -m baselines.run --alg=ppo2 --env=Breakout-ram-v0 --num_timesteps=1e7 --network=mlp

I get an assertion error:

Process Process-1:
Traceback (most recent call last):
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ashish_arartc/anaconda3/envs/baselines/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/vec_env/subproc_vec_env.py", line 7, in worker
    env = env_fn_wrapper.x()
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/cmd_util.py", line 36, in <lambda>
    wrapper_kwargs=wrapper_kwargs
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/cmd_util.py", line 49, in make_env
    env = make_atari(env_id)
  File "/home/ashish_arartc/Notebooks/experiments/synopsys-2019/super_experimental/baselines/baselines/common/atari_wrappers.py", line 229, in make_atari
    assert 'NoFrameskip' in env.spec.id
AssertionError

How can I run this program on the necessary environment?

Thanks in advance for any help.

[pzhokhov]

ah... the culprit is WarpFrame environment wrapper that's added to atari environments - it tries to rescale the observation assuming it is an image; which fails for -ram observations. For now, I'd recommend disabling WarpFrame wrapper here:

https://github.com/openai/baselines/blob/f3a5abaeeb1c1c9136a01c9dbfebc173dc311fef/baselines/common/atari_wrappers.py#L241 I'll keep the issue open until it is patched. Another thing that may be worth considering is that default mlp network (your --network=mlp argument won't work very well with byte observations for two reasons: 1) observation range is between 0 and 255 instead of standard ~0..1 or -1..1 -> this will likely saturate tanh nonlinearities. This is easy by adding normalization wrapper, however: 2) the small changes in ram values may correspond to large changes in state. For instance, life counter increasing by 1 vs life counter decreasing by 1 :) . One way out is to one-hot encode the RAM (this will lead to giant observations though); some code modifications will be needed for that too (for instance, in baselines/common/input.py)