This repository provides the official codes for our paper: Unsupervised Portrait Shadow Removal via Generative Priors (ACM MM 2021).
Unsupervised Portrait Shadow Removal via Generative Priors
Yingqing He, Yazhou Xing, Tianjia Zhang, Qifeng Chen (* indicates joint first authors)
HKUST
In this repository, we propose an unsupervised method for portrait shadow removal, named as ShadowGP. ShadowGP can recover a shadow-free portrait image via single image optimization, without a large paired training dataset, which is expensive to collect and time-consuming to train. Besides, our method can also be extended to facial tattoo removal and watermark removal tasks.
ShadowGP can decompose the single input shadowed portrait image into 3 parts: a full-shadow portrait, a shadow-free portrait and a shadow mask. Blending the three parts together can reconstruct the input shadowed portrait. The decomposed shadow-free portrait is the target output.
To install and activate the environment, run the following commands:
conda create -n shadowgp python=3.7
conda activate shadowgp
pip3 install torch==1.3.1+cu100 torchvision==0.4.2+cu100 -f https://download.pytorch.org/whl/torch_stable.html
pip3 install opencv-python tqdm scikit-image==0.15 Pillow==7.1.0 Ipython ninja
Our code has the same system requirements as StyleGAN2-PyTorch.
Download the pretrained StyleGAN 550000.pt
and the pretrained face segmentation model face-seg-BiSeNet-79999_iter.pth
from the google drive, and put them in checkpoint/
.
cd Shadow-Removal-via-Generative-Priors
mkdir checkpoint
mv ${YOUR_PATH}/550000.pt checkpoint/
mv ${YOUR_PATH}/face-seg-BiSeNet-79999_iter.pth checkpoint/
Optional: If the lpips checkpoint cannot be downloaded automatically, you can also manually download the weights.tar.gz
to the project directory and run tar -zxvf weights.tar.gz
. Then you will see the checkpoint in lpips/weights/v0.1/vgg.pth
.
bash run.sh
Then it will create a results
folder including all the intermidiate results and the final output results in results/res
.
Our code is built on StyleGAN2-PyTorch.
@inproceedings{he21unsupervised,
title = {Unsupervised Portrait Shadow Removal via Generative Priors},
author = {He, Yingqing and Xing, Yazhou and Zhang, Tianjia and Chen, Qifeng},
booktitle = {ACM International Conference on Multimedia (ACM MM)},
year = {2021}
}