Images generated by the StyleGAN of the DatasetGAN different from the original ones (with the same StyleGAN)

[PoissonChasseur]

Hi,

When I ran the training part of the "train_interpreter.py" file and got the image "train_data.jpg", I saw that the generated images (line # 2) are not exactly the same than the original ones (line # 1). This problem also seems to have been present in the case of images shared on your Google Drive (eg "train_data.jpg" from car_20). However for car_20, the difference is quite small and does not really affect the correspondence with the images of the masks.

However, in my case, with 128x128 images and a StyleGAN having a fairly high FID-50k (around 23-24), the difference between the generated images causes the problem that the sections of interest have shifted in the images. Therefore, the labeled parts of the mask images no longer correspond to the correct sections of the generated images and the pixel_classifiers are not trained on the correct information.

Do you know if there is a way to correct this problem of reproducibility of the images generated by the StyleGAN for a given latent code?

Note: the "original" images I generated at the start were with the ".pt" file of the pre-trained StyleGAN (after converting [Tensorflow -> PyTorch]) and also used the same "avg_latent" as the one that has was then used for the DatasetGAN. I also used the same PC for the generation of the "original" images and for the training of the DatasetGAN. So normally, there shouldn't be any problem related to these parts.

---

Update # 1: I saw that fixing the random seeds of Pytorch and Numpy helped make the images generated by the [StyleGAN from the DatasetGAN] look like the original ones. However, this is only true for the first selected images that were with a continuous numbering when I created them.

Also, I don't understand how the fixation of the random seeds could have fixed this, as I don't see any part related to random values in the DatasetGAN (unless it was induced by the additive noise of the StyleGAN).

[arieling]

Sorry train_data.jpg in google drive is misleading. The second row is not using the provided latent code. Please ignore that.

Which 128x128 stylegan are you using?

[PoissonChasseur]

In fact, I have trained my own StyleGAN to generate images of 128x128 pixels. I solved my problem by changing the code of the "prepare_data" function (in "train_interpreter.py") to ensure that we have exactly the same random seeds (for PyTorch) as when creating the original images. This is because without it, the random seed used by StyleGAN's noise layers will not be the same as before, thus causing generated images to differ from the original ones (even if the latent code used is the same as before).

Since it is impossible to capture the exact random seeds used by PyTorch, I add a simple loop where we generate images until we are with the same index of images that those that were originally generated. For example, if the original images we chose to put labels on are the generated image numbers [0, 1, 2, 13, 14, 26, ...] - we just need to make sure that we skip the image numbers that don't correspond to the ones selected - so that in the end, we have the same random seed that we originally used to generate each of these images. Logically, to do this, it is also first necessary to have fixed the random seed of PyTorch during the generation of the original images which were then used to create the masks / labeled images.

It took a bit of code to fix this, but now everything is fine for me at this level.