search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

Disable brains to only train one or a subset at a time #3750

Closed lukemadera closed 4 years ago

lukemadera commented 4 years ago

Is your feature request related to a problem? Please describe. I'm building a game that has multiple MLAgents in it. 2 have already been trained, but they are still part of the game. The v0.11 change included removing the Broadcast Hub. "The BroadcastHub has been deprecated. If there is a training Python process, all LearningBrains in the scene will automatically be trained." While this seems nice at first glance, as far as I can tell, there is no way to turn off training on certain brains anymore. Is this true? In a pure training environment where you create a new scene and only ever have one scene loaded at a time, this probably works fine. But in a game that's almost never the case. I have multiple scenes and am training a new agent (brain) with the existing agents in the scene and running. But I do not want to retrain them. How do I disable one or more brains from being trained?

I really hope this re-training of my existing brains is not going to screw up what they have already learned, but I hoping if I just do not update the .nn brain model, that will not be the case. But either way I do not want to clutter the terminal logs or waste CPU or memory trying to train brains I don't want to train.

Describe the solution you'd like A command line argument to mlagents-learn and / or a config to set which brains to train. If we want to keep the default to train all brains in the scene we can, but we need an array of brains to disable in that case.

Describe alternatives you've considered I tried to disable the game objects with the other brains to remove them from the scene, but that is not working. It seems as soon as they get loaded, the mlagents picks them up and uses them, even if they get SetActive(false) soon thereafter. Much of my code expects these common agents to exist so just disabling them in the editor before running mlagents isn't a viable option. They are core to the game and tightly coupled to it. I can easily disable them via scripts when I want to train, but I need a way to update mlagents to tell it to not try to train these brains in the first place.

Additional context Add any other context or screenshots about the feature request here.

andrewcoh commented 4 years ago

Hi @lukemadera

I believe the 'inference only' behavior type in the behavior parameters script is what you are looking for. Agents with this set should run as inference only and not be trained.

Let me know if this solves the issue.

lukemadera commented 4 years ago

Oh perfect, that did it, thank you!!

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.