Generated images have geometrical pattern

[SanaNazari]

I am training stylegan3 on dermoscopic skin lesion dataset (HAM10000) in conditional mode with 7 classes . After almost 6k iterations I reached fid of 11. I assume 11 is a pretty good number to get. The issue is that when I generate samples there is a diamond or X shape patterns in the images that make the images unrealistic. Here are some samples of the real dataset:

And here are some examples of generated samples:

Any advice on what id causing this or how can I overcome it?

[coxfuture]

What cfg are you using? Stylegan3-r is only for datasets with rotational symmetry.

[SanaNazari]

I am using stylegan3-r. Which cfg is better to use then?

[coxfuture]

stylegan3-t seems to be the recommended config for general use. the other option is stylegan2, but that's just a fallback to the previous ADA methods

btw, stylegan3-t uses less VRAM so you can probably increase your batch size from what it was on -R

[PDillis]

I'd recommend using --cfg=stylegan2 if quality is your main goal. StyleGAN3 configs do not have a higher quality, just more capabilities (in specific, rotational and translational equivariances). If neither of these matter to you, then go for StyleGAN2-ADA, so it's best you know which augmentations to use depending on your dataset size. As per the figure in the ADA paper:

So e.g., if you have around 2k images, then set --aug=ada --augpipe=bgc --target=0.4 as per the figure above. Note that this is done with unconditional training of the models, so perhaps the conditional one you are training might differ.

[SanaNazari]

Thank you for the explanation. I am using 10k images. But I don't see any --augpipe in train options in stylegan3 train.py

[PDillis]

It already is using --augpipe=bgc by default as you can see on this line. Since you have a 10k dataset, as per the previous figure, then it'd be best you set --augpipe=bg --target=0.6. As you note, the vanilla StyleGAN3 code doesn't let you set the --augupipe, so you have to add it yourself or use my repo, where I have done that already, plus some other things.

[Haris-Ali007]

Amazing work by NVLabs. I just have one curiosity. I was watching the generated results on pretrained network. I noted that most of the faces generated by stylegan3-t-ffhq-1024x1024.pkl had some geometric patterns mostly in the areas where different shades merges. What is the reason for this?

[coxfuture]

As I understand it, it's due to the way stylegan3 does feature extraction. I noticed the same on my custom models trained with the same config, for the first 1000 kimg there were noticeable "deep dream"-esque artifacts that were not present on stylegan2.

[Haris-Ali007]

Okay. So @isademigod I guess this was also not discussed in the paper as well considering it is a major bug in the model?