search

Golbstein / Keras-segmentation-deeplab-v3.1

An awesome semantic segmentation model that runs in real time
175 stars 72 forks source link
crf-segmentations keras keras-deeplab keras-implementations keras-subpixel segmentation semantic-segmentation

readme

Keras DeepLab V3.1+

DeepLab V3+ for Semantic Image Segmentation With Subpixel Upsampling Layer Implementation in Keras

Added Tensorflow 2 support - Nov 2019

DeepLab is a state-of-art deep learning model for semantic image segmentation. Original DeepLabV3 can be reviewed here: DeepLab Paper with the original model implementation.

Features

  1. Conditional Random Fields (CRF) implementation as post-processing step to aquire better contour that is correlated with nearby pixels and their color. See here: Fully-Connected CRF
  2. Custom image generator for semantic segmentation with large augmentation capabilities.

New Features That Are Not Included In The Paper

  1. Keras Subpixel (Pixel-Shuffle layer) from: Keras-Subpixel for efficient upsampling and more accurate segmentation
  2. ICNR Initializer for subpixel layer (removing checkerboard artifact) ICNR
  3. Comparisson of the original Deeplab model with my Deeplab+subpixel+CRF
  4. Fast training - transfer learning from paper's proposed model to a better model within ~1 hour with 1-1080Ti GPU
  5. Jaccard (mIOU) monitoring during training process for multi-class segmentation tasks.
  6. Adaptive pixel weights.

Results

I've compared the segmentation visual results and the IOU score between paper's model and mine, as well as the outcome of applying CRF as a post-processing step.

Below depicted few examples:

alt text alt text alt text alt text

And the IOU score amid the classes:

alt text

I didn't receive a significant improvement of the IOU scores, perhaps due to low number of epochs. However I believe this method can eventually outperform the original model for a bigger dataset and more epochs.

Dependencies