Use neural network for semantic segmentation

[andrefdre]

The objective of this task is to explore the possibilities of using semantic segmentation for:

• Detecting signal posts
• Detecting crosswalk
• Detecting lanes
• Use lane detection for reinforcement learning (looking at the future)

[andrefdre]

After training a network using deeplabv3 model with 30 epochs, these are the preliminary results:

It didn't get anything right and the code is slow due to going through all the image to check the label and then color it. Without coloring at best it has 20 fps with coloring it is about 0.1 fps.

[andrefdre]

I was finally able to create training scripts for training a semantic segmentation model, currently it works on the cityscapes dataset and our dataset and has 4 models [DeepLabV3, UNet, SegNet, SegNetV2], the SegNetV2 model was created by me and it is similar to SegNet but the encoder-decoder structure uses the output from the encoder in the input of decoder.

The results using SegNetV2 on Gazebo: https://youtu.be/OsCHjqmsfs4

It's possible to see in the video that the network works best in the straight section, which is expected since the dataset contains 300 images with data augmentation and most of them are on that part. The crosswalk and the parking section isn't well represented in the dataset, which leads to bad performance on those labels. I expect to have much higher performance with a larger dataset.

Current labels:

• background
• crosswalk
• driveable
• driveable_alt
• parking

[manuelgitgomes]

Please change the classe: driveable and driveable_alt to left_lane and right_lane or equivalent