search

masato-ka / airc-rl-agent

AI RC Car Agent that using deep reinforcement learning on Jetson Nano
MIT License
84 stars 23 forks source link

Jetbot RL for collision avoidance. #31

Closed Adnan-annan closed 2 years ago

Adnan-annan commented 3 years ago

Hi, I am trying to use your code and extend it for purpose of collision avoidance. Which means placing a small toy car on track and training jetbot to avoid it and continuing to follow path. have the few questions I am looking for your opinion about.

1) I am using a 0.3 to -0.3 steering angle. Jetbot seems to learn but takes a lot long time to train and training result also not seem to be stable. In your opinion what changes should I make to train jetbot for this task. I already trained VAE model with placing car on the track as well.

2) How to decrease the training time ?

3) Is it possible to train using raw images without VAE ? I suppose it could take longer time to train? You crop the image before feeding to VAE . what could be the effect on training for this task if I don't crop the image and increase the VAE dimension from 32 to lets say 64?

4) Is it possible to add the stop action to the action space? if so, what changes would have to be made ? For example training jetbot to stop at the pavement if there are small toy pedestrians using RL? Does this mean I will have to increase action space to 4 and include throttle speed in it as well ?

Looking forward to your kind reply. Thanks

Adnan-annan commented 3 years ago

I would like to work on extending your code for collision avoidance (of small car placed on middle of track), add stop action so that jetbot can stop when their is some pedestrian on pavement (I have a pavement on my track, so jetbot is required to stop at it if there is some toy person there. ) . Do you think it will increase the action space a lot and significantly increase training time ? How will the VAE model perform in that case? I have trained VAE after placing small car on the track and It can reproduce the image in vae_viewer.ipynb . Do you think I am on the right track ? I am desperately looking for your guidance for this task.

Adnan-annan commented 3 years ago

@masato-ka I am looking for your opinion on these tasks, kindly reply to my questions when you have free time. Thanks

masato-ka commented 3 years ago

This is interesting extension. I think jetbot can take avoid other car. However, I can not prediction that your question. Because In that situation have not been test.(I dont have answer for your question.) But, I try to answer...

A1. As with going off course, I think the only thing you can do is to try to stop the car before you hit it. A2. I think increase that. I can not estimate increase time. A3. Yes it is. But I think training time more increase. In addition change some code for without VAE. If do not crop Image and increase dimention of VAE, increase training time. A4. I think good way is "stop action" implements as other component. Object Detection model can detect pedestrian easily. If you detect it, stop car control. It is not related policy by DRL. That's simple way.

masato-ka commented 2 years ago

I will close in 3 days when do not update.