search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.98k stars 4.14k forks source link

Can one Agent have two different BehaviorParameters compenents ? #6116

Closed lyy0095 closed 3 months ago

lyy0095 commented 4 months ago

Hello, I was training a simple environment for the driving and operating of the excavator Agent. Excavators have their unique characteristics, that is, they must first move to the appropriate position before using the robotic arms for dig operations. So, the training can be divided into two NNModels, one for drive and the other for arms. Considering the observations and actions are different for the Agent, I added two different BehaviorParameters compenents to the Agent, but I found no solutions/functions to switch between the two BehaviorParameters compenents during training?

SetModel function as following is only to replace, not to switch: SetModel("train_driveNet", driveNet); SetModel("train_operationNet", operationNet);

How can I switch between the two BehaviorParameters compenents during training? Thank you for the response!

lyy0095 commented 4 months ago

Moreover, I checked the example Wall Jump scene. The Agent of Wall Jump can train two different NNModels by SetModel function in one BehaviorParameters when the dimensions of observations and actions are the same, respectively. If the dimensions of observations and actions are different in two NNModels, Is there any feasible example code available?

lyy0095 commented 3 months ago

After many tests, the proper answers will be as following:

  1. One Agent can have many BehaviorParameters compenents, but only the first BehaviorParameters can have effects, for the code in Agent.cs line 950 is “m_PolicyFactory = GetComponent< BehaviorParameters >();”.
  2. If you want to train two different NNModels for one Agent, train them as https://github.com/mbaske/robot-ants .