Projecting images using class-conditional GANs

[yulonglin]

Is it possible to project images using a class-conditional GAN in this repo? Looking at the code, there doesn't seem to be support for this.

[hkhailee]

Personally when attempting to project an image into a trained class conditional GAN I receive a AttributeError: 'NoneType' object has no attribute 'ndim' error. This is slightly addressed in #99 but it explains how its happening and doesn't address a solution.

Additionally: It appears even if you project an image into latent space of a non conditional GAN (using the same image data without labels) and then try to generate an image using the conditional GAN with the --projected_w instead of a --seed there is an assertion error, likely to vector size to a "mismatch" network.

[Gass2109]

To project target images using class-conditional GAN, we need to specify the label information in the generator mapping function. That is, we can modify this line in projector.py:

w_samples = G.mapping(torch.from_numpy(z_samples).to(device), None)

to, for example:

w_samples = G.mapping(torch.from_numpy(z_samples).to(device), torch.cat([label]*w_avg_samples, dim=0))

where label is the one-hot encoded vector of the class information (like used in generator.py).

Also, I found that the projection gives acceptable results (with slower optimization) even if we don't specify the label (torch.zeros([w_avg_samples, G.c_dim]).

[yulonglin]

@Gass2109 Thanks! I'll try that

@hkhailee Yes I think the error shows up because None is passed in place of the "conditioning" part of the conditional GAN. In the simplest case, it can be a "one-hot encoded matrix" as shown by the code from @Gass2109

This PR also uses the same general idea: https://github.com/NVlabs/stylegan2-ada-pytorch/pull/79

[tengshaofeng]

@yulonglin @hkhailee @Gass2109 @nurpax you really provide many useful information to me. But When I load weights from "https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/transfer-learning-source-nets/ffhq-res512-mirror-stylegan2-noaug.pkl", and run the projector.py ,The synth_image is different from target image. Why? I need your help. target image:

synth_image:

train loss:

step 994/1000: dist 0.22 loss 0.22 step 995/1000: dist 0.22 loss 0.22 step 996/1000: dist 0.22 loss 0.22 step 997/1000: dist 0.22 loss 0.22 step 998/1000: dist 0.22 loss 0.22 step 999/1000: dist 0.22 loss 0.22 step 1000/1000: dist 0.22 loss 0.22 Elapsed: 75.3 s