Lin-Zhuo Chen, Zheng Lin, Ziqin Wang, Yong-Liang Yang and Ming-Ming Cheng
⭐ Project Home »
The official repo of the TIP 2021 paper `` Spatial information guided Convolution for Real-Time RGBD Semantic Segmentation.
Speed is related to the hardware spec (e.g. CPU, GPU, RAM, etc), so it is hard to make an equal comparison.
I get the following results under NVIDIA 1080TI GPU, Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz:
Model | mIoU(480x640) | mIoU(MS) | FPS(480x640) | FPS(425x560) |
---|---|---|---|---|
SGNet(Res50) | 47.7% | 48.6% | 35 | 39 |
SGNet | 49.8% | 51.1% | 26 | 28 |
SGNet_ASPP | 50.2% | 51.1% | 24 | 26 |
If you want to measure speed on more advanced graphics card (such as 2080ti), you can use the environment of pytorch 0.4.1 CUDA 9.2 to measure inference speed.
Download NYUDv2 dataset and trained model:
Dataset | model | model | model | |
---|---|---|---|---|
BaiduDrive(passwd: scon) | NYUDv2 | SGNet_Res50 | SGNet | SGNet_ASPP |
Put the pretrained model into pretrained_weights
folder and unzip the dataset into dataset
folder.
To compile the InPlace-ABN and S-Conv operation, please run:
## compile InPlace-ABN
cd graphs/ops/libs
sh build.sh
python build.py
## compile S-Conv
cd ..
sh make.sh
Modify the config in configs/sgnet_nyud_test.json
(mainly check "trained_model_path").
To test the model with imput size $480 \times 640$, please run:
## SGNet
python main.py ./configs/sgnet_nyud_test.json
## SGNet_ASPP
python main.py ./configs/sgnet_aspp_nyud_test.json
## SGNet_Res50
python main.py ./configs/sgnet_res50_nyud_test.json
You can run the follow command to test the model inference speed, input the image size such as 480 x 640:
## SGNet
python main.py ./configs/sgnet_nyud_fps.json
## SGNet_ASPP
python main.py ./configs/sgnet_aspp_nyud_fps.json
## SGNet_Res50
python main.py ./configs/sgnet_res50_nyud_fps.json
If you find this work is useful for your research, please cite our paper:
@article{21TIP-SGNet,
author={Lin-Zhuo Chen and Zheng Lin and Ziqin Wang and Yong-Liang Yang and Ming-Ming Cheng},
journal={IEEE Transactions on Image Processing},
title={Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation},
year={2021},
volume={30},
pages={2313-2324},
doi={10.1109/TIP.2021.3049332}
}
Deformable-Convolution-V2-PyTorch
If you have any questions, feel free to contact me via linzhuochen🥳foxmail😲com