search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.13k forks source link

Live Inference when using Imitation Learning #4012

Closed Sebastian-Schuchmann closed 1 year ago

Sebastian-Schuchmann commented 4 years ago

Hello Unity Team,

am I correct in assuming that recording demonstrations and seeing the agent train in realtime isn't a thing anymore? I remember using that feature in an older version of ML-Agents and it was very handy.

If it really doesn't exist anymore, it would be great to see it making a comeback!

shihzy commented 4 years ago

hi @Sebastian-Schuchmann - to confirm, are you referring the the Online (in-editor) behavioral cloning features in the past?

Sebastian-Schuchmann commented 4 years ago

Yes, exactly!

shihzy commented 4 years ago

hi @Sebastian-Schuchmann - we deprecated this features since most games training with BC require more than just one demonstration (and that you can always save the recording and train offline).

With that, could you describe why it would be important to have this in your scenario? We didn't hear pushback when we deprecated and assumed that most users would use the offline mode.

Sebastian-Schuchmann commented 4 years ago

Hey! Thanks for your answer and what you are saying makes sense. To be honest, what I liked about it was mostly the showcasing potential. When presenting ML-Agents in a lecture or a video, it was a great way to quickly demonstrate the ability of imitation learning.

Nunukuluk commented 3 years ago

To follow up on this, I am currently working on a project in which I want to train an agent to combat another, specifically in a sword fight duel. For now, I've used self-play, but I believe it would greatly improve the outcome if the agents could learn from imitating a person duelling in VR. So essentially, have one student sword fighter and one teacher sword fighter (person in VR) combatting each other. However, I dont see a way in which I can use the current imitation learning approach to do so, since meaningful demonstrations can only be done by the teacher if the student starts learning and fighting back during the demonstrations.

I might be wrong to believe the deprecated online behavioral cloning would solve this, but when I look at older documentations for this, it seems like it could. Is there another way to achieve what I'm describing with the current release?

Thanks in advance!

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.