search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.2k stars 4.16k forks source link

NaN in SAC training #3214

Closed mnsmuts closed 4 years ago

mnsmuts commented 4 years ago

After 92k steps an agent training to copy a demonstration starts to report: WARNING:mlagents_envs:An agent had a NaN observation in the environment and show this in tensorboard: image

Environment has 1 agent with 47 observations and 1 stacked vector, space type is continuous and action space is 4.

mlagents v13, windows 10, config file designed to imitate only: image

I have not seen this before, could it be something to do with the upgrade to v13?

chriselion commented 4 years ago

@mnsmuts If there is a NaN in your observations, you should definitely try to find out where it's coming from and fix it. I just added some better NaN checks on the C# side (https://github.com/Unity-Technologies/ml-agents/pull/3221), so if you apply those changes, you should get the callstack for where the observations are being added.

mnsmuts commented 4 years ago

Great thanks

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.