search

yuval-alaluf / SAM

Official Implementation for "Only a Matter of Style: Age Transformation Using a Style-Based Regression Model" (SIGGRAPH 2021) https://arxiv.org/abs/2102.02754
https://yuval-alaluf.github.io/SAM/
MIT License
632 stars 151 forks source link

Use another generator pretrained weight #14

Closed vinduon closed 3 years ago

vinduon commented 3 years ago

Hi sir!

I have a checkpoint file .pkl and then I use Rosinality to convert it into .pt. So I have some confusion:

1) My question is could I use this model for inference by SAM? I knew that when training you saved stylegan-ffhq-config-f.pt into checkpoint sam.pt.

2) Could I load the sam.pt model and change the state_dict of stylegan-ffhq-config-f path to my stylegan model? This idea comes from the fact that you freeze your generator when training, so that the weight of generator does not affect the final SAM weight for inference.

3) Or should I retrain the SAM model with my own generator model?

4) Or should I retrain styleGAN encode in pSp repo?

Thank you so much sir.

yuval-alaluf commented 3 years ago

2. Could I load the sam.pt model and change the state_dict of stylegan-ffhq-config-f path to my stylegan model?

You could try and see what happens during inference. The reason that you will probably get different results is that the SAM encoder was trained with respect to the stylegan-ffhq-config-f model you mentioned. Therefore, if you change the generator, you will change the latent space as well and therefore get different results (most likely). However, this is worth a try since it requires minimal work.

If this doesn't work, then yes, you can try re-training SAM with your generator, perhaps starting from the pre-trained SAM encoder (rather than training from scratch).

vinduon commented 3 years ago

Thank you for your help, sir!

vinduon commented 3 years ago

Oh I have one more question: How many iterations you trained your SAM model, and pSp also? I see that the maximum is 500,000 but maybe it is sooner right?

yuval-alaluf commented 3 years ago

The pSp was taken from the official pSp repository. I don't recall how many iterations we trained pSp. And SAM was trained for about 60,000 iterations.

vinduon commented 3 years ago

Thank you, sir.