search

williamyang1991 / VToonify

[SIGGRAPH Asia 2022] VToonify: Controllable High-Resolution Portrait Video Style Transfer
Other
3.53k stars 442 forks source link

strange colors transfer #8

Closed Cococyh closed 1 year ago

Cococyh commented 1 year ago

I use my own stylegan2 model and train it according to the method you provided, but I get some strange colors in generated photos, especially in the clothing parts.

image image

Can you give me some advice？

williamyang1991 commented 1 year ago

I guess you use the VToonify-T model, right? Since my training is to use the synthetic data from Toonify (your stylegan2 model), which only covers the face regions and parts of the clothing regions, the trained VToonify-T might not be able to infer the right colors in the regions far from the face region during testing.

You can see in my example of VToonify-T, the results also got some color bias in the clothings.

 

A solution might be using a VToonify-D model with color preservation? In the following example of VToonify-D, you can see the color is better preserved.

 

Cococyh commented 1 year ago

yes, I use the VToonify-T model, thanks for your advice, and I will try VToonify-D:)