search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.29k stars 4.17k forks source link

Extending Imitation Learning model catalog with SQIL #5936

Closed stkovacevic94 closed 1 year ago

stkovacevic94 commented 1 year ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] BC algorithm can fall victim to covariate shifts. GAIL algorithm relies on min/max optimization and suffers from stability issues, and is harder to deploy.

Describe the solution you'd like A clear and concise description of what you want to happen. Implementing SQIL algorithm https://arxiv.org/pdf/1905.11108.pdf.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. I have reviewed few offline RL algorithms, but nice thing about SQIL is that it is basically form of regularized BC that eliminates covariate shift problem. Also, its not hard to implement, and have better convergence properties than GAIL.

Additional context Add any other context or screenshots about the feature request here.

stkovacevic94 commented 1 year ago

I'm interesting in implementing this myself, with medium guidance and help from community (mainly to get onboarded, and some pointers in the code where i can add this) since i'm new to Unity ML Agents (but not IL and RL itself).

Taikatou commented 1 year ago

I really like this feature and inclusion and think it would be beneficial

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity.