You can convert text to face image!!! Thanks to @Puzer and @pbaylies for the original style Encoder
Short explanation of encoding approach:
1) Bert is used for transforming sentence to embedding 2) Original pre-trained StyleGAN generator is used for generating images 3) Pre-trained ResNet network is used for transforming a reference image and generated image into high-level features space 4) Simple Model is used for transforming Text Embedding and image into high-level features space
More examples you can find in the Jupyter notebook
You can generate latent representations of your own images using two scripts: 1) Extract and align faces from images
python align_images.py raw_images/ aligned_images/
2) Download the stylegan pre-train model (百度云链接), put the stylegan.pkl to model folder
3) Training ResNet50 encoder
train your own with trainResnet.py or download my pre-trained model! Put the model in model/finetuned_resnet.h5
Origin github
4) Find latent representation of aligned images
python encode_images.py aligned_images/ generated_images/ latent_representations/
You can generate sentence embedding with bert-as-service 1) Install bert-serving
pip install bert-serving-server
pip install bert-serving-server 2) Start the BERT service bert-serving-start -model_dir ./model/chinese_L-12_H-768_A-12 -num_worker=4
bert chinese model download address 百度云链接
You can describe face image by yourself, like this:
48889.png 黑色短发年轻女子,有刘海,开心的笑,露齿,大眼睛
48872.png 黑色短发青年男子,戴墨镜,侧面
Used image from FHHQ dataset
The model is simple with some Dense layer
python trainTextToFace.py