search

manavkataria / behavioral_cloning_car

Using Keras to train a deep neural network for predicting steering angles based on camera input. Trained on a Unity3D simulator.
6 stars 1 forks source link
camera deep-neural-networks keras predicting-steering-angles self-driving-car tensorflow udacity-nanodegree unity3d-simulator

readme

Table of Contents

Behavioral Cloning

Using Keras to train a deep neural network for predicting steering angles based on camera input. Trained on a Unity3D simulator.

TLDR; Watch the Video - Gone in 60 Seconds!

This video contains subtitles mentioning salient points and challenges encountered during the project. Overcoming these is no laughing matter! šŸ˜‰

Youtube Video

Goal

The goals of this project are the following:

Files

My project includes the following files:

Repository includes all required files and can be used to run the simulator in autonomous mode.

Code Quality

Functional Code

Using the Udacity provided simulator and my drive.py file, the car can be driven autonomously around the track by executing

$ python drive.py model.h5

Comments inline with code

The model.py file contains the code for training and saving the convolution neural network. The file shows the pipeline I used for training and validating the model. It contains detailed comments to explain how the code works.

Model Architecture & Preprocessing

Architecture: nVidia End-to-End Deep Learning Network

My model consists of a convolution neural network with 3x3 filter sizes and depths between 32 and 128 model.py

The model includes ReLU layers to introduce nonlinearity. model

Objective, Loss function and Hyper-Parameter tuning

I used Mean Squared Error as a loss metric. It seems reasonably appropriate to train the model to follow the training steering angles to some close enough extent. It was important to not let the loss function go very close to zero. A very low loss indicates memorization and thus overfitting.

The optimization space was non-linear. Hence there were instances where the model training would not converge. Retraining reinitializes the weights randomly and provides another shot at convergence. The model used an Adam optimizer; the learning rate was not tuned manually. I did use Stochastic Gradient Descent initially but Adam is proven to be a better choice for most cases.

Controlling Overfitting

The model contains Dropout layers in order to reduce overfitting. The Dropout was set to 10%

The loss function and predictions were carefully monitored to ensure the loss doesn't go too low and the predictions aren't a perfect match. This ensures the model isn't overfitted. The model was ultimately tested by running it through the simulator and ensuring that the vehicle could stay on the track.

Image Preprocessing

The Image data is preprocessed in the model using the following techniques:

Steering Angle Preprocessing

The steering angles were scaled up to STEERING_MULTIPLIER = 100.

Training Strategy

Building an Overfitted Model with Minimal Data

The overall strategy for deriving a model architecture was to initially overfit the model on three critical images before building a regularized model for the entire track. This saves time and validates the approach. I look at this as a lean design approach or a MVP approach.

I used the following three images, twice for each training (3x2 = 6 per set) and validation set (50% validation split). Thus making a total of 12 images in the initial dataset.

Recovery: Extreme Left of Lane

center_2017_01_16_18_49_00_738

Drive Straight: Center of Lane

center_2017_01_16_18_49_02_100

Recovery: Extreme Right of Lane

center_2017_01_16_18_49_04_959

I ran this for 30 epochs to achieve satisfactory loss convergence.

Loss Function 30 Epochs

The predictions came out close but not extremely overfitted, which is ideal!

Predictions for 12 Frames

Building a Regularized Model with Augmented Data

The next step was to run the model on the entire training dataset (full track). The provided Udacity dataset had 8k images. The label distribution was quite Asymmetric and Unbalanced [Histogram: Asymmetric and Unbalanced]. I used Horizontal Flipping to Make this symmetric [Histogram: Symmetric But Unbalanced]. And lastly, Histogram Equalization for achieving balance in the training dataset [Histogram Equalization: Symmetric and Balanced]. I also added a random 10% noise to the steering angles for each image. This helps avoid overfitting as well.

Raw Data Histogram: Asymmetric and Unbalanced

Histogram: Asymmetric and Unbalanced

Horizontally Flipped Data Histogram: Symmetric But Unbalanced

Histogram: Symmetric But Unbalanced

Fully Processed with Histogram Equalization: Symmetric and Balanced

Histogram Equalization: Symmetric and Balanced

Loss Function 5 Epochs

Loss Function 5 Epochs

Finally, the balanced dataset was randomly shuffled before being fed into the model.

Once the dataset was balanced, the vehicle is able to drive autonomously around the track without leaving the road.

Predictions for 120 Frames

Predictions for 120 Frames

Acknowledgements & References