search

Farama-Foundation / Minigrid

Simple and easily configurable grid world environments for reinforcement learning
https://minigrid.farama.org/
Other
2.05k stars 597 forks source link

[Question] What do you mean by "done" action in the action space? #395

Closed kbkartik closed 9 months ago

kbkartik commented 11 months ago

The documentation doesn't mention the following things:

1) What does "done" action stands for?

2) Where do we check the termination conditions for each of the environments?

pseudo-rnd-thoughts commented 11 months ago

  1. Done is only used in two environments - "go to door" and "go to object" to signal that the agent thinks the task is complete. Otherwise the environment should ignore it

  2. The documentation for each environment has a section called termination providing the different cases - https://minigrid.farama.org/environments/minigrid/

NaphatPRM commented 11 months ago

Can I further ask for the termination of the "go to object"? It says that it'll be terminated from "The agent stands next the correct door performing the done action." What does it mean?

Roihn commented 11 months ago

You may check this part of code to better understand the terminate condition:

https://github.com/Farama-Foundation/Minigrid/blob/9dbdf612351822fadc6a8a4d1c7519043bd9a1de/minigrid/envs/gotoobject.py#L156

I guess the stands next the correct door is a typo, which should be the correct object in this task.

To briefly explain, if the agent takes the "done" action, the environment will check if the agent is nearby the target object (its up, down, left, or right side, no need to face towards the object in this task). If yes, then give a positive reward (self._reward()) and set terminated=True. Otherwise, only setting terminated=True without updating the reward.