stylegan2-ada-pytorch

[olegkorshunov]

Could you please explain to me how to use stylegan2-ada-pytorch weghts? I downloaded the model from here stylegan2-metfaces-1024x1024.pkl, after I followed issue #104 and use colab notebook that gets weight, g_ema in formdict_keys(['g_ema', 'latent_avg']),, after I save like

torch.save(
    {'state_dict':w['g_ema'],
    'latent_avg':w['latent_avg'],
    'opts':{'exp_dir': '', 'dataset_type': 'ffhq_encode', 'encoder_type': 'GradualStyleEncoder', 'input_nc': 3, 'label_nc': 0, 'batch_size': 8, 'test_batch_size': 8, 'workers': 8, 'test_workers': 8, 'learning_rate': 0.0001, 'optim_name': 'ranger', 'train_decoder': False, 'start_from_latent_avg': True, 'learn_in_w': False, 'lpips_lambda': 0.8, 'id_lambda': 1.0, 'l2_lambda': 1.0, 'w_norm_lambda': 0.025, 'lpips_lambda_crop': 0, 'l2_lambda_crop': 0, 'stylegan_weights': '', 'checkpoint_path': None, 'max_steps': '10000', 'image_interval': 100, 'board_interval': 50, 'val_interval': 1000, 'save_interval': 1000, 'resize_factors': None, 'device': 'cuda:0'}
},'./pretrained_models/sg2-metface-1024.pt')

and obviously, it didn't work :blush:, as I understand I need to change the generator? or I just save in incorrect form?

[olegkorshunov]

I used a solution for weight converting from here https://github.com/dvschultz/stylegan2-ada-pytorch, but when I run inference.py pSp couldn't load encoder, so I put in encoder ir_se50 and after I run inference I got one image everywhere, now I don't understand clearly how I can get the encoder, I need to train him with weights stylegan2? or I lost it when converting weights

[yuval-alaluf]

Please see: https://github.com/eladrich/pixel2style2pixel/issues/247

[olegkorshunov]

Sorry could you please clarify for me because I'm a little confused I think I am doing something wrong, I try to run inference.py with stylegan2-metfaces-1024x1024.pkl I converted the weights to the format ->.pt {'state_dict','latent_avg} and my next steps, if I want to run inference.py I need to follow this repo restyle-encoder and since encoder work with a human facial domain meta-face I need for training use only this ffhq-dataset? and for the testing I can use CelebA and as I understood here I need to specify only stylegan2-metfaces model

python scripts/train_restyle_psp.py \
--dataset_type=ffhq_encode \
--encoder_type=BackboneEncoder \
--exp_dir=experiment/restyle_psp_ffhq_encode \
--workers=8 \
--batch_size=8 \
--test_batch_size=8 \
--test_workers=8 \
--val_interval=5000 \
--save_interval=10000 \
--start_from_latent_avg \
--lpips_lambda=0.8 \
--l2_lambda=1 \
--w_norm_lambda=0 \
--id_lambda=0.1 \
--input_nc=6 \
--n_iters_per_batch=5 \
--output_size=1024 \
--stylegan_weights=./pretrained_models/stylegan2-metfaces-1024x1024.pt

or can I take weights for decoder from psp_ffhq_toonify.pt?

[yuval-alaluf]

I'm having a hard time following exactly what you want to do. Let's break it down. You have a metfaces generator that you converted to a pt file. Were you able to load these weights correctly? Can you generate random samples using this generator and do these images seem reasonable? After you can confirm this, let's try to understand what you want to do with the encoder.

If you can do the above, let me try to understand what you're trying to do with the encoder: you have the FFHQ dataset of real faces and you want to encode them into the metface generator?

[olegkorshunov]

Yes I converted metfaces generator to a pt file and I can generate random reasonable images use metfaces generator. Now I want to do image-to-image translation, I want that model work like psp_ffhq_toonify

[olegkorshunov]

Thank you for your amazing work! I started the training and now I'm waiting and I hope that I did everything right :smile: