distillpub / post--differentiable-parameterizations

A powerful, under-explored tool for neural network visualizations and art.
https://distill.pub/2018/differentiable-parameterizations
Creative Commons Attribution 4.0 International
26 stars 12 forks source link

Should we tell more about original motivation behind Weighted Fourier param? #70

Closed znah closed 6 years ago

znah commented 6 years ago

The original motivation came from the fact, that InceptionV1 gradients look similar to white noise for some reason, while natural images have 1/f amplitude spectrum decay.[google: natural image spectrum] At first I was using Lapacian pyramid normalization to fix the gradient spectrum, and then together with Chris we formulated it as optimization in different parameters space (or as Chris states, optimization under a different vector norm).

colah commented 6 years ago

This section now hyperlinks to our discussion in feature vis. That seems like the best way to handle it to me.

image