Traffic light images from Udacity simulator

[yaboo-oyabu]

Collect images to train tl_detector for the virtual environment.

In order to help you acquire an accurate ground truth data source for the traffic light classifier, the Udacity simulator publishes the current color state of all traffic lights in the simulator to the /vehicle/traffic_lights topic in addition to the light location. This state can be used to generate classified images or subbed into your solution to help you work on another single component of the node. The state component of the topic won't be available when running your solution in real life so don't rely on it in the final submission. However, you can still reference this topic in real life to get the 3D world position of the traffic light.

Note that you need to upgrade pillow to 6.2.2 to get messages from /image_color topic.

[CaoFM]

I have created a branch to record camera images and logging truth to a csv file. You need to create this folder manually first if trying to run on your setup. CarND-Capstone/ros/src/tl_detector/light_classification/sim_img

I am attaching a small batch of sample image and the csv file here. Before I run the simulator long enough to create tons more training data, please comment on:

1. If the formatting and info included in the csv file are agreeable and sufficient for the training (image file name, state, distance from the light in number of waypoints)
2. Do we need roughly equal amount of RED, YELLOW and GREEN light (yellow is extremely rare and will take some thinking to gather enough different data points)
3. Do we need 'no light present' data? If so, similar amount compared to light present or?
4. How many total image do we need?

sim_img.tar.gz

[yaboo-oyabu]

@CaoFM Thanks for the great work.

If the formatting and info included in the csv file are agreeable and sufficient for the training (image file name, state, distance from the light in number of waypoints)

I think this is perfect. I like distance from the light, since it allows us to pick smaller or larger traffic light images when sampling training images from larger collection of images.

Do we need roughly equal amount of RED, YELLOW and GREEN light (yellow is extremely rare and will take some thinking to gather enough different data points)

We can use these images to train (1) traffic light classification model and (2) traffic light detection model. Regarding traffic light classification, we may need equal amount of RED, YELLOW and GREEN. However, IMO, this will be done without sophisticated machine learning algorithm as we've done in lane line detection assignments in the beginning of this Udacity course. On the other hand, (2) will require equal amount of traffic light and non-traffic light images, and I think this is not so difficult.

I suggest to generate equal amount of traffic light and non-traffic light images for traffic light detection. If we can't create traffic light classification with OpenCV, then let's revisit this topic.

Do we need 'no light present' data? If so, similar amount compared to light present or?

I think so. Please see the above comment.

How many total image do we need?

That's a tough question. In general, we need less number of images for transfer learning. Since we can sample images later, shall we collect more than 1,000 images for each category (traffic light or non traffic light). In addition, we need some yellow traffic light in collected images, but no need to have the equal amount of green, yellow and red.

@CaoFM Can you make a pull request so that we can have your code in master branch.

[CaoFM]

Thank you for the feedback @yaboo-oyabu . I will make update to store 'no traffic light' data and then make a pull request.

I am thinking about having one single learning algorithm to cover both the detection and classification problem. In another word, we could treat 'No Light' as the fourth class in addition to the 'Red' 'Yellow' and 'Green' class.

It's just a thought and I think we can further explore such possibility once we're in that stage. For now I will just focus on making training data available.