search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.17k stars 4.16k forks source link

Reset in multi-agent environments in UnityEnv #1963

Closed inconst closed 5 years ago

inconst commented 5 years ago

Documentation for env.step() says that

When end of episode is reached, you are responsible for calling reset() to reset this environment's state.

But how can I reset separate environments in case of multi-agent setup?

vincentpierre commented 5 years ago

Hi @inconst I am not sure I understand your question fully. In our definitions, an environment can contain multiple agents. When calling env.reset the environment (an all of the agents it contains) will reset. When the Academy of an environment is Done then the Python env will no longer be able to step and env.reset must be called to continue the simulation. In the case of multi-agent setup, there is still a single environment.

inconst commented 5 years ago

Hi @vincentpierre So whenever any of agents finish its episode, Done is called, right? But other agents may need more time to finish theirs, so they can't be reset yet because env.reset() will reset all agents. Is this handled somehow?

vincentpierre commented 5 years ago

There are 2 levels of Done flags : One on the Agents and one on the Environment / Academy. When the Environment / Academy is Done, everything must reset : Academy and Agents alike. When an Agent is done, it is labeled as Done and the simulation can go on. (The Environment is no Done and can proceed)

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.