williamyang1991 / VToonify

[SIGGRAPH Asia 2022] VToonify: Controllable High-Resolution Portrait Video Style Transfer
Other
3.54k stars 446 forks source link

seems pretrain model is not used #18

Closed JohnnieXDU closed 2 years ago

JohnnieXDU commented 2 years ago

Hi, nice work, appreciate it! But two questions confuse me.

  1. in train_vtoonify_d.py, for pre-training, why save the weights of g_ema in line 172 / 387? Looks g_ema.eval() keep the weights unchanged, and g is the generator should to be trained.

  2. after pre-training, in full train process, looks the pre-trained model is not loaded (from vtoonifu_d_cartoon/pretrain.pth).

Thanks!

williamyang1991 commented 2 years ago

This part updates g_ema with g, so its weight changes.

https://github.com/williamyang1991/VToonify/blob/db57c27b4189023a5330c21b015a8e78cc111b87/train_vtoonify_d.py#L340-L342

This part loads pre-trained model

https://github.com/williamyang1991/VToonify/blob/db57c27b4189023a5330c21b015a8e78cc111b87/train_vtoonify_d.py#L68-L69 https://github.com/williamyang1991/VToonify/blob/db57c27b4189023a5330c21b015a8e78cc111b87/train_vtoonify_d.py#L430-L431

JohnnieXDU commented 2 years ago

thanks a lot :D appreciate it ~

one more question, could you please provide the full pipeline for training all nets from scratch? just a brief flowchart works fine for me, just a little bit confusing since training ALL-models looks tough. :(

williamyang1991 commented 2 years ago

we only has two steps pretrain the encoder (train the green part) and train the full network (train the green and blue parts).

image

JohnnieXDU commented 2 years ago

Thanks! By the way, what does ema in g_ema mean? (non-abbrevation version is?)

williamyang1991 commented 2 years ago

exponential running average https://github.com/rosinality/stylegan2-pytorch/issues/34

JohnnieXDU commented 2 years ago

thanks a lot!