search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.18k stars 4.16k forks source link

I want to train a car but its not learning. #2355

Closed manmeetsingh0013 closed 5 years ago

manmeetsingh0013 commented 5 years ago

Hi All,

I am working on car racing game and i am using ML 0.8.2 version but car is not learning at all. I have created boundary and using RaycastPreception3D but still no progress.

Please help me with that. Thanks, Capture

harperj commented 5 years ago

Hi @manmeetsingh0013 -- as mentioned in our issue template unfortunately our team doesn't have the resources to help support training of user-created environments. Unfortunately there are a lot of factors (observation setup, action space setup, rewards, hyperparameters) which could contribute to your environment not training. One bit of feedback I can give is that it seems your reward is always -1.000; you'll want to ensure your agent has the ability to see how it can affect the environment to increase or decrease reward in order for it to learn.

manmeetsingh0013 commented 5 years ago

Thanks for reply. I just want to tell you that i am taking the reference of the pyramid scene. i m using the same Raycast and decreasing the reward after the steps like this : AddReward(-1f / agentParameters.maxStep) and also moving the car as per pyramid reference. could you please give the explanation that how how many rays require to detect the left and right walls

harperj commented 5 years ago

How many rays you need to effectively detect walls depends a bit on your environment setup. One thing I can suggest is to add a small positive reward for getting closer to the goal if it is something you're able to do. The Pyramids scene is a particularly hard scene to solve particularly because it only gives rewards after solving a complex task rather than giving small incremental rewards for progress.

manmeetsingh0013 commented 5 years ago

Thank for the reply @harperj . I really appreciate your feedback.

manmeetsingh0013 commented 5 years ago

I have one more question it would be great if you can give some input on it. suppose first time learning process it created .nn model and for second time learning if are using same run id will it be taking some learning inputs from 1st .nn model? hope you get my question. Thanks,

harperj commented 5 years ago

Hi @manmeetsingh0013 -- if you are using the same run-id and want to continue training with the same model you'd currently need to use the --load flag to mlagents-learn

manmeetsingh0013 commented 5 years ago

thanks for the help @harperj

chriselion commented 5 years ago

Thank you for the discussion. We are closing this issue due to inactivity. Feel free to reopen it if you’d like to continue the discussion though.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.