Conditional GAN: NUM_CLASS param for image completion task and whats the deal with random noise?

[koutilya-pnvr]

Hi, am just wondering what to use for the NUM_CLASSES variable if i am performing image completion on generic datasets where there is no significance for the class labels. Because the NUM_CLASSES param determines gen_rand_noise() function output. Also, if i am using a conditional gan on an input image what is the whole point of incorporating a random input noise? The input to the GAN should be another image right! I dont think that is highlighted in the code.

[jalola]

Hi, If you don't want to use class labels, how about using GAN without condition (gan_train.py). However, I believe it performs best on datasets where their images are sharing similar features/structure. As you see we trained on bedroom dataset. There, you can create your own noise (incorporating noises etc.) and the model should generate different images.

Does the dataset you want to train contain so many different things (e.g. airplanes, flowers, different animals in one dataset)? Let me know if I misunderstood your question.

[koutilya-pnvr]

If i am not wrong, you are talking about Conditional GAN conditioned on class labels. Something like, generate images of flowers and then your code generates them. But, i am looking into slightly different concept of conditional gan. If you take a look into this recent inpainting paper, they condition the gan on the input and not on the classes. That is very typical in inpainting task (slightly different from synthesis problem). In such case some details change as follows: 1) We no longer use random noise as input since we are conditioning the input to be specific 2) There is an extra loss involved at the generator (Ex: L2 loss) in addition to the already existing WGAN loss

Let me know what you think of it. Thanks a lot for replying instantly though.

[jalola]

Yes, if we want to generate a flower which we've already defined a tag (e.g. 1, 2, 3,...) for this flower class, then we put/condition this tag into a noise/input and based on this tag inside the input, the model will know and generate a flower for us instead of image of other classes. Probably you are looking for something different than the model in this repo. I don't know if this repo https://github.com/akmtn/pytorch-siggraph2017-inpainting maybe the one you are looking for?

[koutilya-pnvr]

Thanks for sharing the repo that performs inpainting. I am more interested in using wgan to do the inpainting task rather than what the paper stated which has a very complex training procedure. What i can do is fork this repo and build it for the inpainting task conditioned on the input incomplete image. Do you have any suggestions?

[jalola]

I don' think this current model is not designed to handle such things you wanted since the Generator means to upsample small size of noise to bigger and bigger representation and finally a fake image.

I actually don't know how to make a inpainting task as an input for G in this model. And even if we are able to create such input layer, we also need to change/re-design middle layers as well.