search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.14k forks source link

All the gpu memory is occupied in the beginning of training #3192

Closed SungbinChoi closed 4 years ago

SungbinChoi commented 4 years ago

All the gpu memory is occupied in the beginning of training if I experiment with mlagents 0.13.0. This phenomenon didn't happen until 0.12.1. I experimented with 3DBall example on CentOS7, tensorflow 1.13.1. I both tried PPO and SAC.

xiaomaogy commented 4 years ago

Do you know if you are using the tensorflow cpu or the gpu version? Also are you using the xvfb or no graphics mode or xserver to render?

SungbinChoi commented 4 years ago

I'm using tensorflow gpu version. I'm experimenting on my local linux machine. It happens in both '--no-graphics mode' is turned on or off.

chriselion commented 4 years ago

This was a bug that was fixed in the 0.13.1 release; see https://github.com/Unity-Technologies/ml-agents/pull/3219 for the fix.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.