Integrating custom envs with rl-baselines-zoo

[question] [feature request]

Am enjoying r-b-z. Wanting to make envs, or wanting to modify them.

I know you can define a custom env as described in link 1 and in link 2.

The problem with link 1 is that you're limited in the number of parameters you can pass, EG can't, to my knowledge, apply Optuna like this:

./run_docker_gpu.sh python -m train.py --algo ppo2 --env MYCUSTOMENV-v0 -n 50000 -optimize --n-trials 1000 --n-jobs 2 \
  --sampler random --pruner median

I'm aware you can define a custom env

import gym
from gym import spaces

class CustomEnv(gym.Env):
  """Custom Environment that follows gym interface"""
  metadata = {'render.modes': ['human']}

  def __init__(self, arg1, arg2, ...):

but I don't believe you can pass EG -optimize --n-trials 1000 --n-jobs 2 random --pruner median to it, and so it's more limited than the first script; I haven't found a way to train.py CustomEnv as it were.

Link 2 I haven't gotten to work with r-b-z.

I think it's like this because r-b-z imports or stores its "internal" gym envs somewhere. (IE, you don't need to import Gym to use r-b-z.) So, I can't go in there and modify or add files.

My question is - where does r-b-z store its internal gym envs? Or is there some way to have r-b-z re-reference its internal gym to a "normal" gym import "outside" of r-b-z. This so that I can go in there and modify directly.

Help would be warmly appreciated!

Kind regards

Hello,

EG can't, to my knowledge, apply Optuna like this:

EG = example given ?

I think it's like this because r-b-z imports or stores its "internal" gym envs somewhere.

The known environments are store in gym "registry", that's what the second link describes. We use gym.make to create the env, that uses under the hood the registry to find the env. If you want to use the rl zoo with a custom env, but without changing the rl zoo code, then you need to create a python package of your env, and then use the --gym-package argument (see README)

If you want an example of custom env as a package, you can take a look at the highway envs or the minigrid envs.

Note: the README will be updated soon (there is a small typo in the command line see https://github.com/araffin/rl-baselines-zoo/issues/27 )

araffin / rl-baselines-zoo

Integrating custom envs with rl-baselines-zoo #30