search

google-deepmind / sonnet

TensorFlow-based neural network library
https://sonnet.dev/
Apache License 2.0
9.79k stars 1.3k forks source link

Is there any convenient way to do spectrum normalization? (Sonnet version: 2.0.0) #193

Open isletennos opened 4 years ago

isletennos commented 4 years ago

Dear Deepminder:

For generative adversarial network, there introduces a normalization called spectrum normalization (ref: https://arxiv.org/abs/1802.05957).

However I find according to the knowledge of me, it is a little hard for me to incorporate it into Sonnet.

This is because spectral normalization changes the weights. But I checked snt.Conv2D, seems the weight is intractable due to internal management.

Old version sonnet seems to have a solution, (https://github.com/deepmind/sonnet/issues/99) but I can't find "custom_getter" and other Solutions in Sonnet version: 2.0.0.

So could I be given some suggestion regarding such a situation? Thanks a lot.

chr1sj0nes commented 4 years ago

Hi @isletennos,

I think snt.custom_variable_getter is what you're looking for.

isletennos commented 4 years ago

Thanks for getting back to me without delay!

I couldn't find in the Sonnet Documentation how to use this modules. What should I do if I only get the w in snt.conv2D and process it like a SN?

o-90 commented 2 years ago

There is an implementation of SpectralNormalizer in the little_gan_on_mnist notebook under examples/