Not able to run GV envs

[amitfishy]

Hi again,

I'm having some issues running experiments with the gridverse envs. I think the problem is in the way the environment is created with the wrapper that adds the latent space on top. In the script asym_rlpo/envs/env_gym.py the function make_gym_env gets called and EnvironmentType.OTHER gets assigned to it. I tried bypassing this by just adding a EnvironmentType.GV when a GV env is used. But now something else is wrong and the latent_space shows None. At this point I'm not sure if I should continue or it is just a small change somewhere else I need to fix?

Thanks Amit

[abaisero]

Hello Amit,

The make_gym_env function in asym_rlpo/envs/env_gym.py is not meant to be called directly by the user, but is a utility that is used internally to create only some types of environments, but not others. A similar role is played by the make_gv_env function in asym_rlpo/envs/env_gv.py; again, this is a utility that is used internally to create only GV environments.

To create any kind of environment that is currently supported, you should use the make_env function from asym_rlpo/envs/__init__.py. Internally, make_env will automatically call the appropriate functions, probably either make_gym_env or make_gv_env depending on some conditions.

I assume this is what is causing the issue, and that it will be solved once you switch to using make_env. If the problem persists, could you please describe the problem in more details, and show a minimal working example and the respective stack trace?

Best,

[amitfishy]

I should have specified that I'm just running the main_a2c.py script and following along from there to find the issue. That's what led me to the specific functions I'm talking about. But yes, it is going through make_env and then it goes through make_gym_env instead of make_gv_env which I believe causes the problem.

python main_a2c.py GV-MemoryFourRooms-7x7-v0 a2c

Loading using gym.make

Traceback (most recent call last):
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/main_a2c.py", line 684, in <module>
    raise SystemExit(main())
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/main_a2c.py", line 634, in main
    runstate = setup()
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/main_a2c.py", line 283, in setup
    algo = make_a2c_algorithm(
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/asym_rlpo/algorithms/__init__.py", line 50, in make_a2c_algorithm
    models = make_models(env, keys=algorithm_class.model_keys)
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/asym_rlpo/models/__init__.py", line 36, in make_models
    models = make_models_gv(env)
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/asym_rlpo/models/models_gv.py", line 93, in make_models
    'agent': _make_representation_models(env),
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/asym_rlpo/models/models_gv.py", line 54, in _make_representation_models
    else GV_Representation(
  File "/home/fishy/python_ws/aais-baisero3/code/asym-rlpo/asym_rlpo/representations/gv.py", line 68, in __init__
    space['grid'].high.max() + 1,
TypeError: 'NoneType' object is not subscriptable

I think I'm supposed to call it using yaml file names instead of the GV gym names? This seems to work ok.

python main_a2c.py ../gym-gridverse/gym_gridverse/registered_envs/gv_memory_four_rooms.7x7.yaml a2c

[abaisero]

Oh I see; interesting, I do think in practice I've always used the yaml file configurations to load GV environments, so this never occurred to me, but yeah there's no reason I can see right now why using the gym id shouldn't work as well. There might be some issues due to the gym id not being specific enough for certain configurations that the GV environments need, e.g., whether to include state representations or not.

For the time being, just use the yaml configuration, I'll see to fix this in the coming days.

[abaisero]

I'll keep this open as a reminder, and for future notes, until it's fixed or there is a conclusion.