Question on Learning a Space of Implicit Functions

[AmitMY]

In section 4.4 of your paper you go into an interesting hypernetwork idea that can generate the siren parameters for a space of "functions" (images in that case).

In section 9 of the appendix, you go more into details, and I specifically care about the part where you predict the siren parameters from the input:

As far as I understand

the weights of a 5-layer SIREN with hidden features of size 256

Are:

W1 ∈ R(2, 256) # maps x,y
b1 ∈ R(256)

W2,W3,W4 ∈ R(256,256)
b2,b3,b4 ∈ R(256)

W5 ∈ R(256,3) # maps to rgb
b5 ∈ R(3)

Total params: 2*256 + 256 + 3*(256*256 + 256) + 256*3 + 3 =198,915

So, do I understand correctly that your hypernetwork takes the input from the convnet, input, and does the following:

h1 = relu(U1*input + c1)
h2 = U2*h1 + c2

Where

U1 ∈ R(|input|, 256)
c1 ∈  R(256)

U2 ∈ R(256, 198915) 
c2 ∈ R(198915) 

This doesn't feel right to me.

[alexanderbergman7]

Yes, your description of the hypernetwork architecture is correct.