search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.12k stars 4.15k forks source link

Periodically Restart Executable Training Option #3280

Closed GabHook closed 2 years ago

GabHook commented 4 years ago

Would be nice to have a command line training option for restarting the Unity executable after every steps. We are not perfect and our executable currently runs out of memory after like 2 million steps. So it would be nice to have a fresh start of the executable after each steps so I could let the training go overnight without worrying it will crash. Probably it wouldnt be hard to add either.

vincentpierre commented 4 years ago

Thank you for your comments. We’ve documented your feedback and will prioritize when appropriate.

vincentpierre commented 4 years ago

For reference, our internal tracking for the issue : MLA-558

hvpeteet commented 2 years ago

While we have not added this feature, we do have a change on main allowing configurable restarts upon environment crashes. https://github.com/Unity-Technologies/ml-agents/pull/5553. Not sure if this fulfills your need but it sounds like it may.

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.