KeyError: 'DeathValley-v0'

[weiguowilliam]

I can run the sample command in Readme.md well. But when I tried to run MountainHike example:

python ./code/main.py -p with environment.config_file=MountainHike.yaml environment.name=DeathValley-v0 algorithm.use_particle_filter=True algorithm.model.h_dim=256 algorithm.multiplier_backprop_length=10 algorithm.particle_filter.num_particles=15 opt.lr=2.0e-04 loss_function.encoding_loss_coef=0.1

It gives me the following error message. Can you help me with that? Thank you.

INFO - POMRL - Running command 'main'
INFO - POMRL - Started run with ID "34"
/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/site-packages/sacred/config/config_files.py:43: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  return handler.load(f)
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/site-packages/gym/envs/registration.py", line 132, in spec
    return self.env_specs[id]
KeyError: 'DeathValley-v0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/weiguopro/Downloads/baselines/baselines/common/vec_env/subproc_vec_env.py", line 15, in worker
    envs = [env_fn_wrapper() for env_fn_wrapper in env_fn_wrappers.x]
  File "/Users/weiguopro/Downloads/baselines/baselines/common/vec_env/subproc_vec_env.py", line 15, in <listcomp>
    envs = [env_fn_wrapper() for env_fn_wrapper in env_fn_wrappers.x]
  File "/Users/weiguopro/Downloads/POMDP/code/envs.py", line 17, in _thunk
    env = gym.make(env_id)

...

KeyError: 'DeathValley-v0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/weiguopro/Downloads/baselines/baselines/common/vec_env/subproc_vec_env.py", line 15, in worker
    envs = [env_fn_wrapper() for env_fn_wrapper in env_fn_wrappers.x]
  File "/Users/weiguopro/Downloads/baselines/baselines/common/vec_env/subproc_vec_env.py", line 15, in <listcomp>
    envs = [env_fn_wrapper() for env_fn_wrapper in env_fn_wrappers.x]
  File "/Users/weiguopro/Downloads/POMDP/code/envs.py", line 17, in _thunk
    env = gym.make(env_id)
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/site-packages/gym/envs/registration.py", line 156, in make
    return registry.make(id, **kwargs)
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/site-packages/gym/envs/registration.py", line 100, in make
    spec = self.spec(path)
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/site-packages/gym/envs/registration.py", line 142, in spec
    raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: DeathValley-v0
ERROR - POMRL - Failed after 0:00:10!
Traceback (most recent calls WITHOUT Sacred internals):
  File "./code/main.py", line 477, in main
    id_tmp_dir, envs, actor_critic, rollouts, current_memory, dvrls = setup()
  File "./code/main.py", line 171, in setup
    envs = register_and_create_Envs(id_tmp_dir) # a list of env, len(envs) = num_process = batch_size
  File "./code/main.py", line 287, in register_and_create_Envs
    envs = SubprocVecEnv(envs)
  File "/Users/weiguopro/Downloads/baselines/baselines/common/vec_env/subproc_vec_env.py", line 71, in __init__
    observation_space, action_space, self.spec = self.remotes[0].recv().x
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/Users/weiguopro/opt/miniconda3/envs/pomdp/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError

[maximilianigl]

Hey, the KeyError usually indicates that the environment wasn't registered with gym, so it can't find it. Looking through the code, I can't find where it gets registered, so I'm wondering whether I removed it when cleaning up the code for publication. I'll have a look over the weekend.

[weiguowilliam]

Thank you for the suggestion! I'll also check the registration of the environment..

[maximilianigl]

Hey, sorry, I didn't have time yet to properly set everything up and debug (finished PhD, new computer, etc..), but I had a quick look and remembered what I did back then. The environment should be registered here: https://github.com/maximilianigl/DVRL/blob/38e896c929ed682e8cb37516aa1afd9f6070df76/code/main.py#L263 I'm wondering whether something goes wrong here. If you want, you could check if execution properly enters that if branch and executes the register function. Another alternative I could imagine is that gym had some breaking changes (wouldn't be the first time) and registering now works differently.

[weiguowilliam]

Thank you! I'll try that.

[weiguowilliam]

Solved.

[shawnshenjx]

@weiguowilliam hey, I bumped into this error as well, how did you solve it?

[AAgha66]

I also bumped into it, I think it's caused by a mismatch with the openAI baseline version, but not sure. @weiguowilliam @shawnshenjx do you maybe have any tips how to solve this ?

[AAgha66]

Just for future reference. I think the problem was caused by SubProcVecEnv, so using DummyVecEnv should be fine. I'm not sure what is exactly causing the issue, didn't have much time to look into it. My suspicion was that there is now a mismatch with newer versions of OpenAI baselines. I tried an older commit of OpenAI baselines c0fa11a which works well.

[maximilianigl]

Thank you AAgha66 for your input on how to avoid it. I'm also still not sure what causes it, but I found another (very dirty and hacky solution) is to paste the following into main.py (I did it at line 78, but shouldn't matter too much):

death_valley_configs = {
    "transition_std": 0.025,
    "observation_std": 0.0,
    "goal_reward": None,
    "goal_position": [0.7, 0.5],
    "goal_radius": 0.1,
    "goal_end": False,
    "outside_box_cost": -1.5,
    "starting_position": [-0.85, -0.85],
    "starting_std": 0.1,
    "max_time": 100,
    "max_action_value": 0.05,
    "action_cost_factor": 0.1,
    "shaping_power": 4,
    "hill_height": 4,
    "box_scale": 10,
}
register(
    id="DeathValley-v0",
    entry_point="environments.death_valley:DeathValleyEnv",
    kwargs=death_valley_configs,
    max_episode_steps=75,
)

Something seems to be wrong with registering the environment in the sub-processes and this makes sure it always happens. WARNING: This hard-codes the environmental hyperparams, in particular the observation_std, which now cannot be changed anymore via the command line!