search

Farama-Foundation / Minigrid

Simple and easily configurable grid world environments for reinforcement learning
https://minigrid.farama.org/
Other
2.13k stars 611 forks source link

the terminal state does not show the goal #177

Closed initial-h closed 2 years ago

initial-h commented 2 years ago

Hi, I want to map states to string keys to get one-to-one mapping. But I find when the agent reaches the goal, the terminal state does not show the goal, this will make the terminal state may be the same as the reset state.

For example in MiniGrid-Empty-5x5-v0, the reset state is reset_state and the terminal state is terminal_state but the two states has the same observation [[[2 2 2 2 2 2 2] [2 2 2 2 2 2 2] [2 2 2 2 2 2 2] [2 2 2 2 2 2 2] [2 2 2 2 2 2 2] [2 2 2 2 2 2 2] [2 2 2 1 1 1 2]]

[[5 5 5 5 5 5 5] [5 5 5 5 5 5 5] [5 5 5 5 5 5 5] [5 5 5 5 5 5 5] [5 5 5 5 5 5 5] [5 5 5 5 5 5 5] [5 5 5 0 0 0 5]]

[[0 0 0 0 0 0 0] [0 0 0 0 0 0 0] [0 0 0 0 0 0 0] [0 0 0 0 0 0 0] [0 0 0 0 0 0 0] [0 0 0 0 0 0 0] [0 0 0 0 0 0 0]]]

How can I make a difference with the two states. Thanks.

maximecb commented 2 years ago

Hmm well normally when the agent goes over the goal state, the episode ends, so it doesn't really matter what the agent is observing there. The done flag should be set to True for that step. Though I would assume that the agent still perceives the goal at the center position when it's over it.

initial-h commented 2 years ago

Thanks for your quick reply! Yeah, usually we don't need the terminal state. I'm doing some improvement based on the terminal state, I need a one-on-one mapping for it. I'm wondering if I can modify it some where.

maximecb commented 2 years ago

Sorry for the delayed response. I was abroad and the wifi was terrible.

The problem you're having is because the object being displayed at the agent's position represents the object which the agent is carrying.

If you comment out this if statement, it should behave the way you would expect: https://github.com/maximecb/gym-minigrid/blob/master/gym_minigrid/minigrid.py#L1190

initial-h commented 2 years ago

Hi, I think this self.carrying = None all the time, the code will not enter the if statement. If I want it to show the goal, can I just set self.carrying to some number like 8?

maximecb commented 2 years ago

Comment out the whole if-else statement.

initial-h commented 2 years ago

This is what I want. Thanks so much!