Include environment options in gym env name?

[zplizzi]

I would like to test out some of these environments in existing RL reference implementations, eg Baselines, Dopamine, RLLib, etc. But I would especially like to be able to use some of the environment options (eg distribution_mode="easy"), and it seems like this isn't super easy to do without digging deep into the source code of these reference implementations and modifying the line where they do gym.make() to add the desired flags.

I wonder if it would be possible to encode the environment options in the environment name itself. For example, instead of just allowing the env name procgen:procgen-coinrun-v0, you could allow procgen:procgen-coinrun-v0@distribution_mode=easy (or any such syntax). That way people could try out these reference implementations without modification even from the command line, but still access the cool environment options you all have enabled.

[christopherhesse]

I think there's too many combinations to do that unfortunately. However if you write some code like this: https://github.com/openai/procgen/blob/master/procgen/gym_registration.py#L24 and then register the names you want to use in your experiment it should work fine.

If you want to do that without modifying the procgen source, you can create your own package, say, procgen_experiments and do gym.make("procgen_experiments:<name>")

If you do that you'll want to register the environments in __init__.py like procgen does: https://github.com/openai/procgen/blob/master/procgen/__init__.py#L10

[zplizzi]

Gotcha, yeah it would have to be a modification in Gym itself to support flags without explicitly registering each combination.

[christopherhesse]

gym does support that through the use of **kwargs to gym.make, but it sounded like you didn't want to do that

[srikarym]

If I pass distribution_mode as an argument to gym.make, would the argument be used while creating the environment? For ex: env = gym.make("procgen:procgen-fruitbot-v0", distribution_mode = 'easy', num_levels=1, start_level = 0)

[christopherhesse]

It's supposed to, does that not work?

[srikarym]

It does. I thought the above comments meant that these args can't be used with gym.make.

[christopherhesse]

gym does support that through the use of **kwargs to gym.make, but it sounded like you didn't want to do that

^ This was meant to indicate that it did. The original poster may not be in a position to use that though, since they mentioned using some RL frameworks, which probably don't support gym.make with kwargs, only the string version.

[christopherhesse]

I updated the docs to make this more clear.