search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.13k forks source link

Making CaptureDeltaTime a parameter in the Academy? #2892

Closed hk1ll3r closed 4 years ago

hk1ll3r commented 4 years ago

I trained a model for my simple slime game here. The agents use only vector observations and no visual input.

In my case it makes sense to have FPS independent of FixedUpdate frequency for inference time. Essentially agents are parts of the physics update. So I modified the Academy code to set Time.CaptureDeltaTime (CDT) to 0 instead of the hard coded magic number 60. This makes the rest of game development and building for different platforms easy... or at least consistent with how I would normally go about changing FPS, quality settings, etc. when ml-agents is absent.

Even for cases where there are visual observations, having the CDT fixed at 60 seems "weird" to me. Where does this magic number come from? Rough estimate of human eye FPS? Wouldn't it make more sense to have CDT as a parameter? For example one could adjust it to 50 FPS to be in sync with physical updates. Or maybe 10 or 100 but not 60.

From reading the Academy code, the changes don't seem too involved. The complications I see with this change would be: 1) setting CDT to 0 and not getting consistent visual observations -> a runtime error could be shown when CDT is 0 and there are agents in the scene with visual inputs. 2) models trained with a CDT would not be fully compatible with other CDT values in case there are visual observations. They might perform really bad or just fine. This again could be addressed with an error or warning at runtime if the current CDT doesn't match that of ALL the active models in the scene.

I'd love to hear MLAgents' teams feedback on this. I'm sure there have been discussions on this at least internally. Are the complications and extra effort to implement not worth having CDT as a parameter?

Alternatively there could be a checkbox to switch between CDT = 60 for when there are visual observations and and CDT = 0 when there are no visual observations. The implementation and complications would be less than the former case.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

hk1ll3r commented 4 years ago

Any updates on this?

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.