Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image

Saikat Dutta, Sourya Dipta Das, Nisarg A. Shah and Anil Kumar Tiwari

Accepted at Mobile AI workshop, co-located with CVPR 2021 Paper | ArXiv | Supplementary | YouTube

Pytorch 1.1.0 Torchvision 0.3.0 skimage 0.16.2

Colab demo

1. Dataset:

Get the EBB! dataset by registering here.

Train split: data/train.csv

Test split (val294 set): data/test.csv

2. Run inference on Val294 set using DMSHN model:

python DMSHN_test.py

3. Run inference on Val294 set using Stacked DMSHN model:

python stacked_DMSHN_test.py

4. To generate PSNR, SSIM and LPIPS scores on output images:

python eval.py -d0 OUT_DIR -d1 GT_DIR --use_gpu 

5. Citation:

@inproceedings{dutta2021stacked,
  title={Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image},
  author={Dutta, Saikat and Das, Sourya Dipta and Shah, Nisarg A and Tiwari, Anil Kumar},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2398--2407},
  year={2021}
}

6. Related work:

[1] Dutta, Saikat. "Depth-aware blending of smoothed images for bokeh effect generation." Journal of Visual Communication and Image Representation (2021): 103089. Paper ArXiv Project page

[2] Das, Sourya Dipta, and Saikat Dutta. "Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020. Paper ArXiv Code

7. Useful Repositories:

[1] SSIM loss

[2] MSSSIM loss

[3] LPIPS