This is the official PyTorch implementation of the AAAI 2021 paper: J. Lin, Y. Yuan, and Z. Zou, MeInGame: Create a Game Character Face from a Single Portrait, the Association for the Advance of Artificial Intelligence (AAAI), 2021.
[Updates]
2021.05.11
: The RGB 3D Face Dataset (Google Drive, a password protected zip file) is now available! Please download and read the LICENSE AGREEMENT carefully, and send the signed license agreement to yuanyi@corp.netease.com via email to get the PASSWORD for the zip file.pip install torch==1.4.0+cu100 torchvision==0.5.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html
pip install opencv-python fvcore h5py scipy scikit-image dlib face-alignment scikit-learn tensorflow-gpu==1.14.0 gast==0.2.2
pip install "git+https://github.com/Agent-INF/pytorch3d.git@3dface"
git clone https://github.com/FuxiCV/MeInGame
cd MeInGame
Prepare the Basel Face Model following the instructions on Deep3DFaceReconstruction, and rename those files as follows:
Deep3DFaceReconstruction/BFM/BFM_model_front.mat -> ./data/models/bfm2009_face.mat
Deep3DFaceReconstruction/BFM/similarity_Lm3D_all.mat -> ./data/models/similarity_Lm3D_all.mat
Deep3DFaceReconstruction/network/FaceReconModel.pb -> ./data/models/FaceReconModel.pb
Download the pre-trained model, put the .pth file into ./checkpoints/celeba_hq_demo subfolder, and the .pkl file into ./data/models subfolder.
Run the code.
python main.py -m test -i demo
# Or
python main.py -m test -i demo -c
# it will run on the CPU, if you don't have a qualified GPU.
Run following command to create training dataset from in-the-wild images.
python create_dataset.py
# You can modify the input_dir to your input images directory.
Download our RGB 3D Face Dataset (Google Drive), unzip it, and place it into the ./data/dataset/celeba_hq_gt subfolder. Please download and read the LICENSE AGREEMENT carefully, and send the signed license agreement to yuanyi@corp.netease.com via email to get the PASSWORD for the zip file. Note: the permission for application is only open to researchers or faculty of universities or research institutes. It is prohibited for students or business entities to apply.
After the dataset is ready, you can train the network with the following command:
python main.py -m train
Please cite the following paper if this model helps your research:
@inproceedings{lin2021meingame,
title={MeInGame: Create a Game Character Face from a Single Portrait},
author={Lin, Jiangke and Yuan, Yi and Zou, Zhengxia},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
year={2021}
}