search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.12k stars 4.15k forks source link

3DBall agents cheat physics because no kinematic Rigidbody #4088

Closed WazWaz closed 2 years ago

WazWaz commented 4 years ago

Describe the bug The 3DBall example moves a collider that doesn't have a kinematic rigidbody. Apparently this allows some strange physics glitch cheat, because after a day's training, the agents were clearly getting fake results. You really need to add a Rigidbody to the 3DBall/Agent and set it as Kinematic to get more reliable physics.

To Reproduce To make the problem "harder" in a different way, I changed the ball into a long capsule and added a grippy physic material, so they had to learn to balance a stick on the top of their heads. After 24 hours of learning, I noticed that quite often the capsule was embedded well inside their head.

I tried to reproduce this bug with the plain ball, but after a day they hadn't discovered the physics glitch. It should be possible for them to find it eventually. I guess that's a problem with this kind of software.

Console logs / stack traces Can provide code and .nn, but the fix is simple - add Rigidbody and set it isKinematic. You may also want to set a Physic Material on the objects, but that's not related to this bug.

Screenshots 2020-06-06

Environment (please complete the following information):

NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.

andrewcoh commented 4 years ago

Hi @WazWaz

Thank you for pointing this out. This is something of which we are aware. It's a holdover/artifact from earlier versions of ML-Agents. For the problem as it is, it doesn't cause an issue because things move slowly enough. However, as you point out, it doesn't support extensibility to more complex variants.

I've logged this internally as issue 1060 and we will discuss a fix. Thank you again!

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had activity in the last 28 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 2 years ago

This issue has been automatically closed because it has not had activity in the last 42 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

github-actions[bot] commented 2 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.