This repository includes the official implementation of IndexNet Matting for deep image matting, presented in our paper:
Indices Matter: Learning to Index for Deep Image Matting
Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2019
Hao Lu1, Yutong Dai1, Chunhua Shen1, Songcen Xu2
1The University of Adelaide, Australia
2Noah's Ark Lab, Huawei Technologies
Our code has been tested on Python 3.6.8/3.7.2 and PyTorch 0.4.1/1.1.0. Please follow the official instructions to configure your environment. See other required packages in requirements.txt
.
We have included our pretrained model in ./pretrained
and several images and trimaps from the Adobe Image Dataset in ./examples
. Run the following command for a quick demonstration of IndexNet Matting. The inferred alpha mattes are in the folder ./examples/mattes
.
python scripts/demo.py
compositon_code.py
to improve the efficiency, included in the scripts
folder. Note that, since the image resolution is quite high, the dataset will be over 100 GB after composition.$PATH_TO_DATASET/Combined_Dataset
├──── Training_set
│ ├──── alpha (431 images)
│ ├──── fg (431 images)
│ └──── merged (43100 images)
├──── Test_set
│ ├──── alpha (50 images)
│ ├──── fg (50 images)
│ ├──── merged (1000 images)
│ └──── trimaps (1000 images)
Run the following command to do inference of IndexNet Matting/Deep Matting on the Adobe Image Matting dataset:
python scripts/demo_indexnet_matting.py
python scripts/demo_deep_matting.py
Please note that:
DATA_DIR
should be modified to your dataset directory;Here is the results of IndexNet Matting and our reproduced results of Deep Matting on the Adobe Image Dataset:
Methods | Remark | #Param. | GFLOPs | SAD | MSE | Grad | Conn | Model |
---|---|---|---|---|---|---|---|---|
Deep Matting | Paper | -- | -- | 54.6 | 0.017 | 36.7 | 55.3 | -- |
Deep Matting | Re-implementation | 130.55M | 32.34 | 55.8 | 0.018 | 34.6 | 56.8 | Google Drive (522MB) |
IndexNet Matting | Ours | 8.15M | 6.30 | 45.8 | 0.013 | 25.9 | 43.7 | Included |
./evaluation_code
folder is used to report the final performance for a fair comparion. We have also implemented a python version. The numerial difference is subtle.Run the following command to train IndexNet Matting:
sh train.sh
--data-dir
should be modified to your dataset directory.If you find this work or code useful for your research, please cite:
@inproceedings{hao2019indexnet,
title={Indices Matter: Learning to Index for Deep Image Matting},
author={Lu, Hao and Dai, Yutong and Shen, Chunhua and Xu, Songcen},
booktitle={Proc. IEEE/CVF International Conference on Computer Vision (ICCV)},
year={2019}
}
@article{hao2020indexnet,
title={Index Networks},
author={Lu, Hao and Dai, Yutong and Shen, Chunhua and Xu, Songcen},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
year={2020}
}
This code is only for non-commercial purposes. As covered by the ADOBE IMAGE DATASET LICENSE AGREEMENT, the trained models included in this repository can only be used/distributed for non-commercial purposes. Anyone who violates this rule will be at his/her own risk.