search

NVlabs / stylegan3

Official PyTorch implementation of StyleGAN3
Other
6.44k stars 1.13k forks source link

Training on high-frequency domain? #65

Open GLivshits opened 3 years ago

GLivshits commented 3 years ago

Hi. I have successfully trained your model on 2 domains, but have a problem with training the quite peculiar one: the fingerprints. This is a high-frequency domain by its nature, and the generator just cannot generate anything even of the shape of a fingerprint (256 resolution, after 100k iterations). I've tried such tricks as stronger r1 regularization, dual-contrastive loss, and ease of the FIR filtering on the fly (higher cutoffs/band halves). I wonder is it even possible to train such domain, or this architecture is just not capable of doing so because there are no obvious coarse features to which finer ones can attach.

stefaj commented 3 years ago

Increase resolution and train longer?

image

PeterL1n commented 2 years ago

Man, training fingerprint seems scratchy. I wonder what's the application lol.

Joke aside, I think StyleGan3 by default should be able to generate high-frequency detail. It has shown to be perfectly fine with human hairs.

You can also adjust the architecture. For example, adding more critical sampling layers at the end, or decreasing the attenuation at each layer? You can even use critical sampling at every layer to trade-off aliasing. You should also test it with stylegan2 to compare.

Tollanador commented 2 years ago

Man, training fingerprint seems scratchy. I wonder what's the application lol.

Upscaling and/or cleaning up an image print of a real-world lifted fingerprint, in order to better match the lifted print with a database of known prints? This would probably be fine if the initial lifted print isn't too degraded, too much information loss though, would mean potentially generating a print of someone else. I do hope a cut-off value is hardcoded to prevent this.

Or maybe it's for more nefarious use, such as re-creating a working, ID matching fingerprint, from a degraded real-world lifted print, using the layer transfer trick to maintain the spatial dimensions of the prints and finger, or something like that?...

Then I suppose there is the Art reason .. although CGI of finger-prints seems like a pretty niche art topic lol

Oh and then there is the straight up academic purpose of 'can it be done', for no other reason than to learn from it.

Now I'm curious as to what other potential use a fingerprint GAN could be used for :)