search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

What are the various ways of integrating trained ML Agent brain with a system such as a unity project. #3146

Closed rajatpaliwal closed 4 years ago

rajatpaliwal commented 4 years ago

Hi all, I am looking into the various ways in which we can integrate a trained ml agent brain with a unity project. For eg. If I want the trained brain to provide guidance for performing a particular behavior , how can I achieve that. I know of one way that is by creating two agents in the same environment with one agent being controlled by the user and the other agent controlled by the trained brain. The agent controlled by the trained brain will not perform any action, it will only display the actions need to be taken using textActions. But I am really not sure of the steps required to perform such procedure. If someone has performed such operation with mlagents trained brain, kindly suggest. Any other approach for achieving the similar functionality is also welcomed. Thanks, Rajat

surfnerd commented 4 years ago

Hi @rajatpaliwal, One way an agent could provide guidance is for you to train an agent to actually perform the behaviors you want and then during inference you could send the user’s input to the neural network and give them feedback however you choose. You’d just need to hook the brain into whatever UI components you want to use to display to the user what the brain thinks they should be doing based on the user’s state.
So instead of modifying transforms or something, you could translate that into text like “move left” or “open door”, if that makes sense. Basically translating your action space to text on your own.

rajatpaliwal commented 4 years ago

Hi @surfnerd , I think from your response , what you mean is to create a let's say Python API to provide user's input to the neural network and output the result in the form of text instructions on the screen. If you can elaborate more on your idea in terms of implementing it in the Unity project, it would be great.

rajatpaliwal commented 4 years ago

Hi @surfnerd , can you elaborate how we can display to the user what the brain thinks based on the user's state.

surfnerd commented 4 years ago

Hi @rajatpaliwal, I was thinking of something very simple, such as mapping it's action space to text. If you are using a discrete action space, each action could be translated to text, manually, by you. As of right now, we don't have anything that does this.

The way I imagined it was that you would train an agent on your environment and then, if you like the results, perhaps you could use that as your "guidance helper" for your user based on the user controlled agent.

For example I you had an agent that moved (up, down, left, right) that solved mazes, you could train it, then put the model into unity. Then based on your user's agents observations, you could suggest an action to the user by requesting an action from the trained model.

If you were to use a discrete action space you could then map these actions to text:

Agent Action Space: [ 0, 1, 2, 3]

Array of text for action: ["up", "down", "left", "right"]

I'm not sure how easy/hard this would be. Does the concept make sense?

Like I said, we don't have any code to support this at the moment, but in theory, you could write the code to accomplish this.

rajatpaliwal commented 4 years ago

Yes, the concept makes sense. Even I was thinking something similar to start with. What I am thinking is once the training is done , I have the trained brain. I will put the model into unity. In the agent script in the AgentAction method , where the discrete actions are present , I will replace those actions for a particular movement with a command to display a particular text instruction on the screen. That way for a particular user state the trained brain will try to take a discrete action, by calling it from a particular branch. But instead of trained brain able to take an action it will be displaying text. @surfnerd would you suggest any improvement or correction.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.