search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.99k stars 4.15k forks source link

BUG? gym wrapper's step() returns "BatchedStepResult" instead of "dict" for the fourth param. #3133

Closed weiziqian closed 4 years ago

weiziqian commented 4 years ago

the 4th. return parameter ("info") for function step() should be "dict"; however in a recent update (2019 dec 10) it was changed into "BatchedStepResult".

which at least caused troubles when training with baseline, e.g.: File "/xxx/stable-baselines/stable_baselines/td3/td3.py", line 343, in learn maybe_ep_info = info.get('episode') AttributeError: 'BatchedStepResult' object has no attribute 'get'

chriselion commented 4 years ago

Thanks @weiziqian - this does look like a bug. I've logged it as MLA-486 in our internal tracker, and we should have a fix for it soon.

chriselion commented 4 years ago

I believe https://github.com/Unity-Technologies/ml-agents/pull/3136 will fix this, but I want to run it by some other folks that are more familiar with the gym interface.

chriselion commented 4 years ago

@weiziqian - this is fixed on master branch now, and will be in the next release. Thanks for the report!

chriselion commented 4 years ago

I also logged a longer-term issue for us to add better testing around this; we have a few unit tests but no integration tests that run training with the gym wrapper.

weiziqian commented 4 years ago

hi Chris, thank you so much for such a prompt response!

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.