This project contains the code (Note: The code is test in the environment with python=3.6, cuda=9.0, PyTorch-0.4.1, also support Pytorch-0.4.1+) for: LEDNet: A Lightweight Encoder-Decoder Network for Real-time Semantic Segmentation by Yu Wang.
The extensive computational burden limits the usage of CNNs in mobile devices for dense estimation tasks, a.k.a semantic segmentation. In this paper, we present a lightweight network to address this problem, namely LEDNet, which employs an asymmetric encoder-decoder architecture for the task of real-time semantic segmentation.More specifically, the encoder adopts a ResNet as backbone network, where two new operations, channel split and shuffle, are utilized in each residual block to greatly reduce computation cost while maintaining higher segmentation accuracy. On the other hand, an attention pyramid network (APN) is employed in the decoder to further lighten the entire network complexity. Our model has less than 1M parameters, and is able to run at over 71 FPS on a single GTX 1080Ti GPU card. The comprehensive experiments demonstrate that our approach achieves state-of-the-art results in terms of speed and accuracy trade-off on Cityscapes dataset. and becomes an effective method for real-time semantic segmentation tasks.
├── datasets # contains all datasets for the project
| └── cityscapes # cityscapes dataset
| | └── gtCoarse # Coarse cityscapes annotation
| | └── gtFine # Fine cityscapes annotation
| | └── leftImg8bit # cityscapes training image
| └── cityscapesscripts # cityscapes dataset label convert scripts!
├── utils
| └── dataset.py # dataloader for cityscapes dataset
| └── iouEval.py # for test 'iou mean' and 'iou per class'
| └── transform.py # data preprocessing
| └── visualize.py # Visualize with visdom
| └── loss.py # loss function
├── checkpoint
| └── xxx.pth # pretrained models encoder form ImageNet
├── save
| └── xxx.pth # trained models form scratch
├── imagenet-pretrain
| └── lednet_imagenet.py #
| └── main.py #
├── train
| └── lednet.py # model definition for semantic segmentation
| └── main.py # train model scripts
├── test
| | └── dataset.py
| | └── lednet.py # model definition
| | └── lednet_no_bn.py # Remove the BN layer in model definition
| | └── eval_cityscapes_color.py # Test the results to generate RGB images
| | └── eval_cityscapes_server.py # generate result uploaded official server
| | └── eval_forward_time.py # Test model inference time
| | └── eval_iou.py
| | └── iouEval.py
| | └── transform.py
pip3 install -r requirements.txt
Env: PyTorch_0.4.1; cuda_9.0; cudnn_7.1; python_3.6,
Clone this repository.
git clone https://github.com/xiaoyufenfei/LEDNet.git
cd LEDNet-master
├── leftImg8bit
│ ├── train
│ ├── val
│ └── test
├── gtFine
│ ├── train
│ ├── val
│ └── test
├── gtCoarse
│ ├── train
│ ├── train_extra
│ └── val
For help on the optional arguments you can run: python main.py -h
By default, we assume you have downloaded the cityscapes dataset in the ./data/cityscapes
dir.
To train LEDNet using the train/main.py script the parameters listed in main.py
as a flag or manually change them.
python main.py --savedir logs --model lednet --datadir path/root_directory/ --num-epochs xx --batch-size xx ...
python main.py -h
python main.py --savedir logs --name lednet --datadir path/root_directory/ --num-epochs xx --batch-size xx --decoder --state "../save/logs/model_best_enc.pth.tar"...
more details refer ./test/README.md
Method | Dataset | Fine | Coarse | IoU_cla | IoU_cat | FPS |
---|---|---|---|---|---|---|
LEDNet | cityscapes | yes | yes | 70.6% | 87.1% | 70+ |
qualitative segmentation result examples:
If you find this code useful for your research, please use the following BibTeX entry.
@article{wang2019lednet,
title={LEDNet: A Lightweight Encoder-Decoder Network for Real-time Semantic Segmentation},
author={Wang, Yu and Zhou, Quan and Liu, Jia and Xiong,Jian and Gao, Guangwei and Wu, Xiaofu, and Latecki Jan Longin},
journal={arXiv preprint arXiv:1905.02423},
year={2019}
}