search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.14k forks source link

Automatically save the *.nn brain model during training #2263

Closed roboserg closed 4 years ago

roboserg commented 5 years ago

Right now I have to stop the training with CTRL+C in the console to get the current brain model. See how the model performs and continue training. Two features that would save time:

  1. Please add an option to automatically save the .nn model during training. Currently you already save some checkpoint information every 50.000 steps. Please save a .nn model as well.

  2. If the feature above is implemented it would be more then awesome to have a separate instance of the environment running during training at real time speed showing agents with the latest brain. For example when training with 6 instances of the environment the agents train with 100x speed. It would be very helpful if there would be a separate instance run in real time with the lastest saved *.nn brain so that the behavior can be observed.

Right now I have to constantly stop the training to observe the agents, it's very time consuming. Thanks.

shihzy commented 5 years ago

hi @roboserg - I think it makes sense to be able to create a frozen graph pb file from the meta, index, and data files that are created with each save frequency. And then create a .nn file to be used with the inference engine. we have it in our backlog. would you be interested to work on this and create a PR? If so, I can ensure we provide support to review it. Otherwise, might take some time for us to work on this given some other priorities.

roboserg commented 5 years ago

nice, thanks! Is the source for the mlagents-learn.exe in this repo? That's the executable that does the training and saving the models, right? (sorry for naive questions)

shihzy commented 5 years ago

correct, you can start here in the trainers - https://github.com/Unity-Technologies/ml-agents/blob/develop/ml-agents/mlagents/trainers

If you make any PRs, please use the develop branch.

harperj commented 4 years ago

Thank you for submitting this request. I’ve added it to our internal tracker with the ID MLA-81. I’m going to close this issue for now, but we’ll ping back with any updates

chriselion commented 4 years ago

.nn files are now saved at the same time as checkpoints (see https://github.com/Unity-Technologies/ml-agents/pull/4127). This will be in the next release (sometime in August).

roboserg commented 3 years ago

You are awesome, thanks

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.