search

leehe228 / LogisticsEnv

UAV Logistics Environment for Multi-Agent Reinforcement Learning / Unity ML-Agents / Unity 3D
MIT License
79 stars 11 forks source link

Question about step() in gymwrapper5 #1

Open xiaoxuh opened 2 years ago

xiaoxuh commented 2 years ago

Very awesome work! While I have some question According to ML-agents, terminal steps contain information about agents which facing the final step. I found the t_s in your code is not used. And another question, is the file gymwrapper5 indeed utilized gym wrapper of unity? I remember a doc in unity, saying gym wrapper in unity is not supported multi agents

leehe228 commented 2 years ago

I'm sorry for the late reply.

  1. In the case of terminal_step in Unity ML-Agents, the state of the agent that has reached the terminal step in the multi-agent environment, that is, the state of the agent that no longer takes action, enters terminal_step. (On the contrary, if the agent does not reach the terminal step and needs to continue action, the state is entered as decision_step.) This is because in our environment, there is no agent who can reach the terminal step because the step limit is not set for individual agents. We are aware of this and plan to add and modify it.

  2. That's right. The official Repository of Unity ML-Agents does not yet support gym wrappers for multi-agent environments. Therefore, I developed and used a generic gym style wrapper that also supports multi-agent environments. It probably works in most multi-agent environments.