pytorch-deep-image-matting
This repository includes the non-official pytorch implementation of deep image matting.
Performance
model |
SAD ↓ |
MSE ↓ |
Grad ↓ |
Conn ↓ |
link |
stage0-paper |
59.6 |
0.019 |
40.5 |
59.3 |
|
stage1-paper |
54.6 |
0.017 |
36.7 |
55.3 |
|
stage0-our |
56.01 |
0.0173 |
33.71 |
57.57 |
|
stage1-our |
54.42 |
0.0175 |
35.01 |
54.85 |
download |
stage1-our-skip |
52.99 |
0.0171 |
31.56 |
53.24 |
download |
- Lower metrics show better performance.
- Training batch=1, images=43100, epochs=12 for stage1-our-skip model, it takes about 1 days.
- Test maxSize=1600.
- GPU memory >= 10GB
Updates
- 2020.09.09: run the demo.py with the latest model (stage1-skip-sad-52.9.pth), and update the visualization results.
- 2020.09.09: adopt VGG-16 backbone with skip connection, and we get better performance with less training costs (12 epochs), Get Stage1-Skip-SAD=52.9
- 2019.10.29: conduct stage0 experiment using current code. Get Stage0-SAD=56.0.
- 2019.09.09: conv6 kernel size from 1x1 to 3x3. Get Stage1-SAD=54.4. The performance of stage1 is as good as paper. While using model released before this day, please change the kernel_size=1 and padding=0 of conv6 in file core/net.py.
- 2019.08.24: Fix cv2.dilate and cv2.erode iterations is set default = 1 and set triamp dilate and erode as the test 1k tirmap (k_size:2-5, iterations:5-15). Get Stage1-SAD=57.1.
- 2019.07.05: Training with refine stage, fixed encoder-decoder. Get Stage2-SAD=57.7.
- 2019.06.23: Training with alpha loss and composite loss. Get Stage1-SAD=58.7.
- 2019.06.17: Training trimap generated by erode as well as dialte to balance the 0 and 1 value. Get Stage0-SAD=62.0.
- 2019.04.22: Input image is normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] and fix crop error. Get Stage0-SAD=69.1.
- 2018.12.14: Initial. Get Stage0-SAD=72.9.
Installation
- Python 2.7.12 or 3.6.5
- Pytorch 0.4.0 or 1.0.0
- OpenCV 3.4.3
Demo
Download our model to the ./model
and run the following command. Then the predict alpha mattes will locate in the folder ./result/example/pred
.
python core/demo.py
Training
Adobe-Deep-Image-Matting-Dataset
Please concat author for available.
MSCOCO-2017-Train-Dataset
Download
PASCAL-VOC-2012
Download
Composite-Dataset
Run the following command and the composite training and test dataset will locate in Combined_Dataset/Training_set/comp
and Combined_Dataset/Test_set/comp
, Combined_Dataset
is the extracted folder of Adobe-Deep-Image-Matting-Dataset
python tools/composite.py
Pretrained-Model
Run the following command and the pretrained model will locate in ./model/vgg_state_dict.pth
python tools/chg_model.py
Start Training
Run the following command and start the training
bash train.sh
Test
Run the following command and start the test of Adobe-1k-Composite-Dataset
bash deploy.sh
Evaluation
Please eval with official Matlab Code. and get the SAD, MSE, Grad Conn.
Visualization
Running model is Stage1-Skip-SAD=52.9, please click to view whole images.
Image |
Trimap |
Pred-Alpha |
GT-Alpha |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Disclaimer
As covered by the ADOBE IMAGE DATASET LICENSE AGREEMENT, the pre-trained models included in this repository can only be used and distributed for non-commercial purposes.