search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.21k stars 4.16k forks source link

Add mlagents-learn system architecture documentation #5447

Closed melvintzw closed 1 year ago

melvintzw commented 3 years ago

Is your feature request related to a problem? Please describe. I've been thinking about how I can go about modifying existing RL implementations or implementing new RL implementations within ml-agents. I started looking through the python codebase and quickly found myself getting quite lost within the layers of abstraction and the numerous modules. It took me a while to see how the CLI arguments are being translated down the software hierarchy to generate the pytorch networks.

Describe the solution you'd like It would be great to have a system architecture diagram describing the component interactions or a contributor's quickstart/guide so that interested people can figure out how to implement or modify the algorithms. This also means that the community can help add/debug algorithm implementations to ml-agents.

Describe alternatives you've considered One alternative is for researchers and ML engineers to build the algorithm and training system from scratch with the Python UnityEnvironment API. This would mean more granular control and visibility over the RL algorithm. However it would mean not being able to make use of all the great work that has already been done by the ml-agents team in building the ml-agents software.

Many thanks for the great work being done w ml-agents!

ervteng commented 3 years ago

This is definitely something we want to do - there aren't currently fixed APIs with the Trainers as they're not really meant to be public. I've logged it to our internal tracker, will keep you posted if we decided to expose this documentation.

in the meantime, if there's something specific you'd like to build, feel free to reach out to us here or at the Unity forums for some pointers on how to do so.