williamyang1991 / DualStyleGAN

[CVPR 2022] Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Other
1.61k stars 249 forks source link

About train a output size of 256 dualstylegan model #61

Closed echo1993in closed 1 year ago

echo1993in commented 1 year ago

I want to train a dualstylegan model using 256 stylegan model .I train a psp model in Z pluz and finetune stylegan in 4*8 batch in 8 GPUs. Destylizing and pretraining dualstylegan is fine.But in finetuning dualstyle ,the result is not well .I try the chageing batch size ,iters ,L2_leg_loss ,perc_loss,but it'is not work. Is any thing wrong in training?There are some results . finetune-000000 finetune-000600 I add img_gen by myself at third. Cartoons_00002_01 Cartoons_00011_01 dualstylegan-000700 In more than 1000 iter,the face is collapse . dualstylegan-001300

williamyang1991 commented 1 year ago

It seems that everything looks right before the last two steps.

Is the penultimate image the pretrained dualstylgan's results? I think it looks like it almost tends to mode collapse (some obvious artifacts). You can use less iterations for pretraining.

Another reason may be that the best training settings of 256 are very different from 1024. I haven't tried on 256. So I'm sorry that I have no idea what kinds of hyperparameters should be used. Maybe you should stop before 1000 iterations?

echo1993in commented 1 year ago

It seems that everything looks right before the last two steps.

Is the penultimate image the pretrained dualstylgan's results? I think it looks like it almost tends to mode collapse (some obvious artifacts). You can use less iterations for pretraining.

Another reason may be that the best training settings of 256 are very different from 1024. I haven't tried on 256. So I'm sorry that I have no idea what kinds of hyperparameters should be used. Maybe you should stop before 1000 iterations?

The penultimate image is training at 600 iterations. I try training illustration style ,it performs well, but not in cartoon style by the same way.

williamyang1991 commented 1 year ago

It's true that large deformation (cartoon) is difficult to learn than small deformation (illustration).

Maybe You can use less deformed pairs to train? For example, use the psp results as the destylized faces.

image

echo1993in commented 1 year ago

I trained the model in less iterations .But the effect is more like a real person, and is not cartoon enough. cartoon_transfer_0_006750_0_overview cartoon_transfer_0_019706_0_overview

williamyang1991 commented 1 year ago

https://github.com/williamyang1991/DualStyleGAN#optional-latent-optimization-and-sampling Latent Optimization is another way to reinforce cartoon effects. You may try it.

echo1993in commented 1 year ago

Thank you so much!