search

PierreExeter / rl_reach

RL Reach is a platform for running reproducible reinforcement learning experiments.
https://rl-reach.readthedocs.io/en/latest/index.html
42 stars 9 forks source link

evaluate_policy.py - Where to put customEnv for gym registration? #2

Closed stefanwanckel closed 3 years ago

stefanwanckel commented 3 years ago

When attempting to evaluate the my her-td3 model I get the following error:

(rl_reach) stefan@stefanwanckel:~/Documents/Masterarbeit/rl_reach_ur5e/code$ python evaluate_policy.py --exp-id 11 --n-eval-steps 100 --log-info 1 --plot-dim 2 --render 1 CURR DIRECTORY: /home/stefan/Documents/Masterarbeit/rl_reach_ur5e/code CURR DIRECTORY: /home/stefan/Documents/Masterarbeit/rl_reach_ur5e/code Evaluating env ur5e_reacher-v1 with algo her and seed 0... Traceback (most recent call last): File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/gym/envs/registration.py", line 121, in spec return self.env_specs[id] KeyError: 'ur5e_reacher-v1' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "enjoy.py", line 233, in env_kwargs=env_kwargs, File "/home/stefan/Documents/Masterarbeit/rl_reach_ur5e/code/utils/utils.py", line 229, in create_test_env vec_env_kwargs=vec_env_kwargs, File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/stable_baselines3/common/env_util.py", line 102, in make_vec_env return vec_env_cls([make_env(i + start_index) for i in range(n_envs)], vec_env_kwargs) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 25, in init self.envs = [fn() for fn in env_fns] File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 25, in self.envs = [fn() for fn in env_fns] File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/stable_baselines3/common/env_util.py", line 77, in _init env = gym.make(env_id, env_kwargs) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/gym/envs/registration.py", line 145, in make return registry.make(id, kwargs) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/gym/envs/registration.py", line 89, in make spec = self.spec(path) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/gym/envs/registration.py", line 131, in spec raise error.UnregisteredEnv('No registered env with id: {}'.format(id)) gym.error.UnregisteredEnv: No registered env with id: ur5e_reacher-v1 Traceback (most recent call last): File "scripts/plot_training_1seed.py", line 35, in df = pd.read_csv(log_dir + 'stats.csv') File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 610, in read_csv return _read(filepath_or_buffer, kwds) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 462, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 819, in init self._engine = self._make_engine(self.engine) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 1050, in _make_engine return mapping[engine](self.f, **self.options) # type: ignore[call-arg] File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 1867, in init self._open_handles(src, kwds) File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/parsers.py", line 1368, in _open_handles storage_options=kwds.get("storage_options", None), File "/home/stefan/anaconda3/envs/rl_reach/lib/python3.7/site-packages/pandas/io/common.py", line 647, in get_handle newline="", FileNotFoundError: [Errno 2] No such file or directory: 'logs/exp_11/her/ur5e_reacher-v1_1/stats.csv' The environment specified is missing! Please update gym_envs/widowx_env/envs_list.csv. Exiting... Traceback (most recent call last): File "scripts/plot_episode_eval_log.py", line 17, in FILE_PATH = str(list(Path(LOG_DIR).rglob('res_episode_1.csv'))[0]) IndexError: list index out of range

I suppose that the main error is: KeyError: 'ur5e_reacher-v1' and the remaining errors are followup errors.

Is it necessary to put the environment folder (in my case ur5e_env) containing the environment class, urdf files, etc into site-packages/gym/envs or is there a way to add my environment to the registry from the original folder structure?

I would be happy to get some help. Thanks.

PierreExeter commented 3 years ago

Hi Stefan,

Yes this error is due to your environment not being registered. I'm assuming that you know how to implement custom Gym environments. If not, please have a look here.

There is no need to modify site-packages/gym/envs to add your custom environment to rl_reach. Here are the steps you should follow:

  1. add your environment class in this directory
  2. register your environment in init.py
  3. cd to this directory and re-install the local pip package with pip install -e .
  4. Test if you can make a new environment with this script 
    import gym
import widowx_env
env = gym.make('ur5e_reacher-v1')

Hope this helps

Note that the step() function in your custom environment should return the same info as in the widowx env, otherwise some evaluation scripts might return an error.

stefanwanckel commented 3 years ago

Thanks for your help! It worked.

PierreExeter commented 3 years ago

Glad to hear that