search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.18k stars 4.16k forks source link

Curriculum learning doesn't work - and no warning is shown - when behavior is not set #4326

Closed Ihsees closed 4 years ago

Ihsees commented 4 years ago

Is your feature request related to a problem? Please describe.

From the docs: Training-ML-Agents.md behavior: Specifies which behavior is being tracked. There can be multiple behaviors with different names, each at different points of training. This setting allows the curriculum to track only one of them.

This sounds like this parameter is optional.

However, if I don't specify this parameter, the next lesson will not be activated.

Describe the solution you'd like My environment has two "behaviours" in it, although one of them is set to "Heuristic-Only". So I haven't tested if the code behaves more rationally if there is only one behaviour.

Ideally it would either use the only behaviour for tracking there is (if there is only 1 in the scene), or give me a warning about a misconfigured config.yaml.

anupam-142857 commented 4 years ago

Hi @Ihsees could you please provide some more details so that this can be answered accurately.

Environment name? Unity version? OS? ML-Agents version? Tensorflow version?

It would be great if you could share the config.yaml you are using.

Hsgngr commented 4 years ago

What should it change if you dont define a parameter in terms of Curriculum learning ? I think you are misunderstanding the concept of it. You need to define values on your own for each lesson.

Ihsees commented 4 years ago

Haven't tested the commit on my codebase yet, but that if-clause looks exactly like what I've had in mind. Thank you!

Edit: Read the conversation on that commit and hadn't considered the cons til now. I don't have a problem with "no default-behavior". I just would like to get some indication that my config will probably not work as I intended.

chriselion commented 4 years ago

This was fixed in #4346 a few months ago (not specifying a behavior will now raise an exception), and is available in the 0.20.0 (and later) version of the mlagents package.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.