Multiple styles

[longnhatne]

Hi, great work !! I cloned the code and run it with the provided pre-trained models and it worked perfectly. However, I tried to train Vtoonify-D with multi-styles dataset and it didn't work. Firstly, I fine-tuned the DualStyleGAN and got the pretrained model and exstyle_code.npy. Then, I trained VToonify-D with provided scripts: python -m torch.distributed.launch --nproc_per_node=N_GPU --master_port=PORT train_vtoonify_d.py \ --iter ITERATIONS --stylegan_path DUALSTYLEGAN_PATH --exstyle_path EXSTYLE_CODE_PATH \ --batch BATCH_SIZE --name SAVE_NAME --pretrain and python -m torch.distributed.launch --nproc_per_node=N_GPU --master_port=PORT train_vtoonify_d.py \ --iter ITERATIONS --stylegan_path DUALSTYLEGAN_PATH --exstyle_path EXSTYLE_CODE_PATH \ --batch BATCH_SIZE --name SAVE_NAME # + ADDITIONAL STYLE CONTROL OPTIONS And the results in the log folder look weird:

The used dataset has 54 different styles and 3 to 4 images for each style. Did I do it wrong, or VToonify can not handle that kind of dataset this time?

Thank you.

[williamyang1991]

The first row should be the images generated by DualStyleGAN. And the following two rows are the results of VToonify to approach the first row. If the first row (the training target) is incorrect, VToonify cannot learn anything.

Have you checked whether your DualStyleGAN is correctly trained?

[longnhatne]

Regarding to training DualStyleGAN, I run the finetune_stylegan.py script and destylize.py, then I got this results. After that, I used the finetune-xxxx.pt checkpoint and the obtained exstyle_code to train the VToonify. Is that the right way?

[williamyang1991]

finetune-xxxx.pt is the finetuned StyleGAN, not DualStyleGAN. You need to further train DualStyleGAN following https://github.com/williamyang1991/DualStyleGAN#progressive-fine-tuning This will gives you a model named generator-xxxx.pt

finetune-xxxx.pt is for training VToonify-T generator-xxxx.pt is for training VToonify-D

[longnhatne]

Ah gotcha, thank you for your support !