search

danielroich / PTI

Official Implementation for "Pivotal Tuning for Latent-based editing of Real Images" (ACM TOG 2022) https://arxiv.org/abs/2106.05744
MIT License
905 stars 114 forks source link

Edit direction #6

Closed bingxumu closed 3 years ago

bingxumu commented 3 years ago

Hi, Thanks for your great work!

Here I get one question, when you fine-tune the parameters of the pre-trained StyleGAN model and use InterFaceGAN methods to edit the image, do we need to find a new semantic direction based on the current parameter model? Intuitively, changing the model parameters will change the semantic direction found by InterFaceGAN.

In general, after you fine-tune the pre-trained StyleGAN model, do you retrain the semantic direction found by InterFaceGAN?

Thanks

danielroich commented 3 years ago

Hi No need to find new semantic directions. All the editing methods such as InterfaceGAN, GANSpace, StyleCLIP and more are working the same. The editing directions that I have uploaded were trained on the pretrained StyleGAN, without PTI.

The key idea is that the fine-tuning happens on the already similar image to the input image. As it turns out, this enables the fine-tuning to make small changes in its weights. Thus altering only appearance-related weights.

Furtheremore, the regularizaion forces the weight change to happen only in local region not affecting other identities.

To summarize, PTI is orthogonal to any editing method. If an editing method works on the pretrained StyleGAN, it will work on the personalized StyleGAN after PTI.

bingxumu commented 3 years ago

Got it. Thanks!