NVIDIA-AI-IOT / jetracer

An autonomous AI racecar using NVIDIA Jetson Nano
MIT License
1.06k stars 319 forks source link

Jetracer Road Following Live Demo Does Nothing #145

Closed CamRoni1339 closed 10 months ago

CamRoni1339 commented 11 months ago

After running through basic motions and the interactive trainer (training 150 images and 20 or so epochs) (both notebooks working) I save it to the path road_following_model.pth, run through the road following notebook and nothing happens. It just drives straight, not adjusting for the track at all. It is almost as if the training is not working and it is just running through basic motions (even though I shut down that notebook). I tested the model live with the blue circle and it is predicting the correct spots, just not doing anything in the road following notebook. I have tried re-training it with new data but I can’t seem to get anything to work. I’m not sure if I am doing something wrong, if it is a hardware issue, or a software issue.

Help would be much appreciated

Thank you

jaybdub commented 11 months ago

Thanks for reaching out.

If the model is predicting the circle correctly, it's likely the issue may not be with the model itself, but with the code used to convert the detection into a control command.

Is the steering working properly with the other notebooks?

One thing that may help test, is to disable the throttle, and hold the racer an inch or two above the track, tilting it side to side. If it's working properly you should see the steering wheels turn in a way that makes sense. If perhaps, the steering is inverted or has a large delay, this could help identify that problem.

Hope this helps, let me know if you have any questions.

John

CamRoni1339 commented 11 months ago

The steering is working properly in the previous notebooks (after some troubleshooting issues). I don't think its a problem with the car, it seems as if it's just not using the trained data.

CamRoni1339 commented 11 months ago

@jaybdub Do you have any other suggestions? I've been stuck on this for a while and would really appreciate it.

jaybdub commented 11 months ago

Hi @CamRoni1339 ,

It may be good to try the original PyTorch model, to eliminate the possibility that there is an issue when using the optimized TensorRT model.

Could you try using the original model PyTorch model (like that at the top of the optimize_model notebook) inside the road following notebook to check if this works?

Best, John

CamRoni1339 commented 11 months ago

Hi @jaybdub

BTW I would like to have it be known that I am very new to this kind of stuff and this is my first project like this. I thought it would be a cool high school project to pick up, but as you can tell I have run into some problems.

I tried running it with the original model and saw no difference. I will list the steps that I went through just to make sure that I did it correctly.

  1. I ran these two blocks

    image
  2. I then skipped over these 3 (because they used the optimized model)

    image
  3. And then I ran these 2

    image
  4. And finally ran the last one

    image

The car still just moves straight with no signs that it is trying to stay on the track. It could be something very simple such as a wrong piece of code, but I am not getting any errors, which just makes this harder. Is there anything I can do to validate the code is loading and running the trained model? I really appreciate all your help.

Thank you so much, Cameron

CamRoni1339 commented 11 months ago

@jaybdub

After working on and figuring out some things, I have solved some problems, but some are still there.

I decided to start over and re-flash the software on a new sd card to make sure that I hadn't messed up any code. After running through the basic motions, training 250 pictures for 15 epochs, and then running the road following notebook, the results are different. The car is still not able to stay on the track properly, but it now shows signs of trying to turn instead of just driving straight. I have looked at the evaluation and the blue dot is in the correct places, it is just still not working in the road following.

Any help would be much appreciated.

Thanks, Cameron

CamRoni1339 commented 10 months ago

SOLUTION

The car.steering_gain value is set by default to -0.65 and you need to set that back to positive. Because of the way the equation is set up the steering will always be backwards of what you want it to be (this was why my car was not following the training. It was trying to, but the gain was making it go the opposite direction). Once you are able to set the steering_gain to positive the car should work normally.