search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.14k forks source link

Logging Std of Reward to Tensorboard #2903

Closed AsadJeewa closed 4 years ago

AsadJeewa commented 4 years ago

Is there an option to display the std of reward in Tensorboard? Please advise on the fastest way to isolate this value

ervteng commented 4 years ago

Not currently but it would be trivial to add. In ml-agents/mlagents/trainers/trainer.py there is a line that reads stat_mean = float(np.mean(self.stats[key])) followed by summary.value.add(). You can just add another summary value add with np.std instead of np.mean.

AsadJeewa commented 4 years ago

Thank you. It works. Please advise on why there are 2 trainer.py files? Working on Windows. My understanding is that one is the github clone and the other is part of the site-packages in anaconda through pip install. But the duplication of files is frustrating. Made the mistake of changing the wrong trainer.py file at first.

ervteng commented 4 years ago

Hi @AsadJeewa, if you're planning on modifying trainer.py, install ml-agents with pip install -e ./ in the ml-agents-envs and ml-agents directories, rather than pip install mlagents, this will force mlagents-learn to point to the files in your github clone. The duplication has to do with how pip works - it makes a copy of the file in the site-packages folder.

AsadJeewa commented 4 years ago

Thank you. Much appreciated

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.