why no validation?

[nellyyi]

I am wondering why you haven't implemented any validation code for the cifar hypernetwork experiment? Is there no danger of overfitting, or is overfitting something you want to do?

Thank you for your code!

[titu1994]

This is a Representation learning model, not a classic discriminative model. The entire goal is to fit the training sample into the weights of the network, conditioned on the pixel coordinates. There is 1 network per image sample. In order to add more images, we use Hypernetworks that generate weights for networks per sample.

But the concept of overfitting does not particularly apply here - we want to overfit the single image per network so that the network weights represent that image as perfectly as possible.

[nellyyi]

First, thank you for your answer!

I totally understand how validation doesn't make sense in the case where you are training on one sample (like the inpainting example). However, in the case of the CIFAR hypernetwork experiment, since you train on all of CIFAR isn't the goal to generalize and to be able to predict the parameter for any image, even for those that the hypernet hasn't seen before? So if we do not use validation the hypernet will overfit to the training set? Thanks again!

[titu1994]

That's precisely what the Cifar script does. It trains and overfits the train set. The we evaluate only once on test set.

Cifar 10 is a very varied dataset, unlike the ones in the paper with homogeneous faces. Samples from Cifar 10 come out blurry and uneven in the test set images.