search

VincentXWD / create-girls-moe-pytorch

An implementation of paper from NIPS2017 'Towards the Automatic Anime Characters Creation with Generative Adversarial Networks' using pytorch.
MIT License
81 stars 10 forks source link

readme

Create Girls Moe

This repo contains a PyTorch from-scratch implementation of the paper Towards the Automatic Anime Characters Creation with Generative Adversarial Networks.

What The Paper Did

Generating 二次元(にじげん) MOEst avatars...

generated avatars from my pre-trained model. Some avatars seem strange. I need to add some tricks and continue training it.

Environment

Attention

This is an unfinished repo. I'm training the models and completing the README.md under src >.<
If you want to use this repo, I strongly recommand you to read codes carefully.

Networks' Structure

I want to call it a DRAGAN-like SRGAN structure because I use the gradient penalty as the paper told and two SRResNet as discriminator and generator. The SRResNet(modified as the paper described) are like this:

I have some modifications in this structure:

  1. I weighted label's loss and tag's loss with half of λadv beacause the loss described in the paper was so hard-core for me. (More details please refer in src/model/gan.py)

  2. Remove the sigmoid operation in adversarial loss calculating since the results with sigmoid layer may cause some problems.

  3. Using Multi-Label Soft Margin Loss for tags' loss calculating.(Cross Entropy Loss may better because of the imbalance of the images' tag distribution. But I don't have too much time for weights tuning. :-D )

Data Preparing

  1. Cause I built an extremely clean dataset for this task. I'm glad to share my data-Preparing method here.

  2. Crawled the images from the website as the paper proposed. Read the readme and codes in src/dataset/Spider/ to get more information.

  3. I used the lbpcascade_animeface model for face detecting. source codes are in src/dataset/FaceDetect/

  4. illustration2vec was used for face tagging. Please check the files in src/tag/

  5. Remove the invalid images manually.

Generative Adversarial Network

  1. The discriminator and generator were defined in src/model/networks/.
  2. The training strategy of GAN was written in src/model/gan.py.

Tools for statistics

  1. Currently I have some simple tools for face data statistics. More details in src/statistics/dataset/.

Super Resolution Processing

Coming soon.