search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

About the imitation learning #2151

Closed guoguoguilai closed 5 years ago

guoguoguilai commented 5 years ago

I want to train a agent to play tennis using imitation learning in a VR game. Another toolkit I use is VRTK. I wonder how to override AgentAction() since the position and rotation of the rack are control by the VR controler. Can someone give me a hint?

xiaomaogy commented 5 years ago

AgentAction() is where you get the action from the model and instruct your own agent to do the corresponding thing. If your agent is controlled by VR controller, you don't need to override the AgentAction(), you should create your own playerbrain asset and map your action in there.

guoguoguilai commented 5 years ago

AgentAction() is where you get the action from the model and instruct your own agent to do the corresponding thing. If your agent is controlled by VR controller, you don't need to override the AgentAction(), you should create your own playerbrain asset and map your action in there.

Thank you for your reply. I have two questions about this:

  1. What does creating my own playerbrain mean? Just create a Play Brain in the Project scene and add brain parameters?
  2. In my project, there is no Key useful. As a player, I don't need to click any button on the VRcontroller. All the tracking stuff are made automatically by VRTK. So how can I map my action? Thank you in advance.
xiaomaogy commented 5 years ago

@guoguoguilai

  1. You can refer to https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Create-New.md#add-brain-assets for how to create a brain. One of the option is player brain.
  2. You will have to map the tracking stuff's action in the player brain yourself.
guoguoguilai commented 5 years ago

@xiaomaogy I am sorry about disturbing you so mush. I am still confused about how to override AgentAction() and map the key in PlayerBrain.

  1. As far as I am concerned, the AgentAction() takes the actions provided by the trained model. And this AgentAction() will be called in both PlayBrain and LearningBrain. You said I don't have to override this AgentAction(). If I follow your suggestion, how can my student agent take action without a VRController?
  2. About the continuous inputs. I can understand the method of maping inputs. I found that all the examples are like: press a key ,then add force to change the position. But for VR, I don't know how to edit the continuous inputs. What is the input of VR?
xiaomaogy commented 5 years ago

Hi @guoguoguilai

  1. Sorry what I said earlier was confusing. I meant that you don't need to override the AgentAction() separately for your PlayerBrain. You do need to implement your own AgentAction() logic, which maps a certain action output from a model (or from a VR controller) to a certain actual action(like a force applied on a certain direction).

  2. I'm not familiar with the VR input signal, I guess you can do something like: press a key, then change the position of a certain gameobject in a certain direction?

guoguoguilai commented 5 years ago

@xiaomaogy I guess VR input signal is kind like the mouse movement. But I don't know how to map this. I already find how to map VRcontroller keys, but I cannot find the controller movement in unity docs. https://docs.unity3d.com/Manual/OpenVRControllers.html. I am still looking for solution...

ervteng commented 5 years ago

Thanks for the comments - closing this issue due to inactivity. Feel free to open a new issue if you're still running into problems.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.