What is the hardware requirement ?

[earcz]

Hi, Firstly, I have read your work and truly loved it. Thank you for sharing ! I will dig into that.

I would like to implement your codes and grasp the idea behind it. As I understand from the compute power, my personal computer should have TitanX GPU, 32 gb ram and Intel Core i7-6700K CPU @ 4 GHz x 8 at leas to work. If this is correct, why did you use TitanX GPU ? I am asking because you used Floydhub.com for training.

Also, which hardware did you use in Lincoln MKZ to run entire autonomous modules ? (e.g. Nvidia Drive-PX and etc). All in all, can you please define the entire system requirement as hardware for testing your codes ? Best.

[teeekay]

Training was done on Floydhub, but also replicated on local machines (I used a 1060GTX with 6GB).

The TitanX GPU, 32 gb ram and Intel Core i7-6700K CPU @ 4 GHz x 8 specifications represent the computing hardware that was present in the car - which was set up by Udacity (not us). I am not aware of the details, but some general information is in this post - https://medium.com/udacity/were-building-an-open-source-self-driving-car-ac3e973cd163.

I was also able to run the project against the simulator (https://github.com/udacity/CarND-Capstone/releases) successfully with my GPU. There were occasional latency issues when trying to run it on laptops with less powerful GPUs, and in virtual machines.

you might also want to look at implementations by other groups like: https://github.com/SiliconCar/CarND-Capstone

[earcz]

First, thank you for the response and related links that you provided. That will help a lot.

Did training take too long when you used your GPU ? Because I have a 1070GTX with 8GB and am curious about training time difference between your GPU and Floydhub. Do you remember how long did it take with 1060GTX ?

[teeekay]

It did not take long to train. It looks like It was running at about 6 seconds per epoch, which would mean that 300 epochs ran in about 1800 seconds on the 1060GTX. Looks like it took twice as long on Floydhub.

[earcz]

Lastly, which training(s) did you carry out ? (e.g. Object detetion, traffic light classification and detection etc.)

[teeekay]

Our initial method, which we stuck with, was general traffic light classification based on the input image -identifying red, yellow, green, or none. This worked well for the simulator which has consistent lighting, and where the lights are generally against a solid background (blue sky or green hills). We ran into problems using this model in the variable lighting and background conditions seen on the outdoor test track. The classifier was not reliable. I was able to increase reliability by a) adding images of the traffic light under different lighting conditions and retraining the model, and b) by running inference on clips of the image (effectively zooming in) in order to make any traffic light larger within the image.

I also tried using the Tensorflow object detection API, and found based on our image library that it was a reliable method to find the traffic light in the image. But at that point I did not want to change the method we were using if I could make the other method work. After the fact, I think that detection and then classification is probably a better solution.

[edufford]

@earcz Thanks for your interest in our team's project.

As some additional background, this repo is the result of our development to complete the final educational project for the Udacity Self-Driving Car Nanodegree program, and so it is focused on the specific hardware and test course that Udacity had set up to allow us to see our code driving their MKZ vehicle on their closed test course (small open area parking lot with a single traffic light) for demonstration purposes only.

As @teeekay mentioned, the neural network training was limited to simple traffic light classification from a single forward camera to keep the project scope within reason for the team's available resources (small data set of reference images, team member remote locations without physical access to real vehicle, etc). However, even with such a simple classification task, we could experience how challenging it can be to get good robustness for real world conditions compared to a simulator each time Udacity tested our code in the car.

More general object detection was not part of the project though, and the test course was clear of any other objects around the car. The vehicle's LIDAR was also only used for localization to their pre-mapped lot area to be able to follow the set waypoints around the course.

Even though the project was focused specifically on Udacity's requirements, hopefully it can still be helpful in some way for your purposes. Best of luck!

[earcz]

@teeekay I did not understand just this part of your comment. Could you please paraphrase it ?

But at that point I did not want to change the method we were using if I could make the other method work. After the fact, I think that detection and then classification is probably a better solution.

@edufford I truly thank you for your time to explain all these things to me and for good wishes. Could you please provide the Udacity's requirements (e.g. hardware setup for simulation or something else) that are not mentioned in your guide if exists and possible? Therefore, I can grasp the concept and build a solid background to do something by myself.

[teeekay]

@earcz We were able to improve simple light classification sufficiently for our needs in the project. We did not need to change to using other methods like object detection/classification which might be a better general solution.

[earcz]

@teeekay I understand now, thank you.

[edufford]

@earcz If you are working towards running the code along-side the simulator on your local machine, then any relatively modern PC that can run native Ubuntu 16.04 linux with ROS Kinetic and the Dataspeed DBW SDK should be fine. I definitely recommend running native linux though, trying to use a virtual machine or even Docker can still have too much processing overhead to run in realtime.

The Udacity v1.3 simulator is made with the Unity game engine and has some options for graphic quality settings that you can adjust to be able to keep lag low enough to run in realtime. The only heavy compute requirement is when enabling the camera image feed (a checkbox in the simulator) for the neural net to do inference processing, but since you have a 1070GTX GPU it should be no problem as long as TensorFlow GPU acceleration is set up.

The other compute specs you originally asked about, such as the TitanX GPU etc, were just what Udacity had set up as the in-car PC for their MKZ to run projects from all of the student teams so it may be more powerful/generic than really necessary, and the requirements would depend on your specific application.

[earcz]

You provided quite clear explanation that I need. Thank you so much !