Fixed maze every episode, Random agent location

[subho406]

Hey, I'm looking to modify this environment slightly. I'm looking for an environment where the maze does change after every call to the reset (it is fixed based on the seed used to initialize it), but the player location is reset randomly every reset call. Is it possible to do in this in the current implementation or will it require some code changes? Do you have some advice on what changes might be required to get this working.

[jurgisp]

Sorry for delayed reply. You mean the maze does not change, but the player location does?

It would require some code changes for sure. Some pointers:

• The place in our code where maze is regenerated on episode init is in MemoryMazeTask.initialize_episode_mjcf
• It calls the underlying MazeWithTargets.regenerate() which is in dm_control codebase. But it could be overriden in MazeWithTargetsArena which inherits from that.
• That code actually relies on the TextMazeVaryingWalls.regenerate, which inherits from a labmaze package, and is responsible for generating a text-based grid maze, that specifies schematically locations of the walls, spawn positions, object positions (the dm_control MazeWithTargets just turns that textual representation into a MuJoCo model). So this would probably be the place to tweak that.

Alternatively, have you considered keeping the concept of episode <=> single maze, but just spawning the agent in a random location each time it collects a reward? That's what DMLab mazes do. That would be an easier change, I think, and perhaps conceptually cleaner. Because if you keep the maze unchanged over different episodes, then is there a meaning to an "episode" anymore?

Or, if you do want to keep layout of the maze always the same and be that part of the environment definition, then how about just not generating at all, and drawing your own fixed maze? That could also be done by defining your own TextMaze which inherits from labmaze.RandomMaze.

[subho406]

Spawing the agent in a random location is a great idea, do you know how that is done in the current code? and how I can find the possible agent agents I can to?

[jurgisp]

Try something like

mjcf.get_attachment_frame(self._walker.mjcf_model).pos = (spawn_x, spawn_y)

inside if target.activated ... clause, where self._pick_new_target() is called.