Tutorial for ML-Environment example creation.

[Smileynator]

We currently have example projects for ML-Agents, settings that are explained, code examples on how to setup and run an environment. I am however missing one step, and that is how you come from "setting up an agent" to "have the agent successfully train".

Even in what i perceive as simple games, i run into issues where i am unsure what is holding the agent back from perfecting it's gameplay. It seems to learn all the concepts of the game, but then can't put it together enough to come to the right conclusions. Where i am uncertain what options or information might even be missing. Learning effectively plateaus where i think it shouldn't.

For this reason, it might be helpful if some of the example projects, came with a step-by-step on why the approach used, is the right approach for the job.

• What is an example where stacked vectors are mandatory for learning to function properly?
• What is a use case for more than 1 hidden layer? (i know in flat data categorization, but i can't wrap my head around, and can't find resources on how you would view game sensor data as such data categorization problems)
• What are typical situations where the agents just has too little information to complete it's job? (EI: If the agent can't perceive an obstacle, it cannot avoid it. But if an obstacle is moving erratically, how can the agent properly make that out from sensory data?)

Typical environment/sensor setup and the reasonings behind them for the examples we have, i think would achieve this. Or at least help deducing how more complex agents or scenarios would have to be tackled.

[hughperkins]

I'm just a bystander, i.e. zero affiliation with Unity, but I am attempting to make youtube videos/tutorials, on reinforcement learning. I'm looking for potential ideas/inspiration. It seems like you created this ticket several months ago, and probably figured out the answers to all your questions by now? But if that's not the case, what are your thoughts on having a Zoom call sometime, so that I can find out more about the problems and challenges you are encountering?

[Smileynator]

Not much has changed actually. I did not have more time to sink into the issue, and most of the answer seem to boil down to "start to study more on how machine learning actually works" Things like how dataset distribution works, how to train for edge cases, and how to balance out a neural network. That last one seems to be more "guesswork" as in, tweak some values, see if it improves training time or efficiency, stick with what works unless the performance starts to deteriorate.

It goes deep, and i don't have the time to do a legitimate deep dive on the subject sadly. Hope you find what you are looking for though, i am sure most people run into the same hurdles once they actually start to apply ML.

[hughperkins]

That last one seems to be more "guesswork" as in, tweak some values, see if it improves training time or efficiency, stick with what works unless the performance starts to deteriorate.

Yes and no. Yes, figuring out what works is typically an empirical question. But 'no', because there are things you can look at to decide what to tweak next. How do you know if your model is failing because it's under-capacity? under-regularized? not exploring enough? etc. (For the last point, you can see for example my video about entropy regularization https://studio.youtube.com/video/1ppslywmIPs/edit ).

Happy to Zoom to look into your specific problems, might give me inspiration for new videos :)