search

zsyzzsoft / co-mod-gan

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Other
444 stars 67 forks source link

Image Sequence Noise #39

Closed bananajoe182 closed 2 years ago

bananajoe182 commented 3 years ago

Hi, first of all thanks you for this brilliant work!

Is there a way to use this for a sequence of images with some kind of temporal consistency ? I`m looking for a way to force the generator to always sample from the same latent noise so that the outputs are always the the same. Like a predefined seed to be able to regenerate the same results. Any idea? Would this also be possible with an already pretrained model ?

Many Thanks!

zsyzzsoft commented 2 years ago

I think maybe not very possible without extra training...

bananajoe182 commented 2 years ago

Ok, thanks. But if I train a custom dataset any idea where I would need to modify the model to generate images from the same latent noise and not randomly ?

zsyzzsoft commented 2 years ago

With the pre-trained model, you can also run inference using the same latent noise by making this line a constant. However, my concern is that even if the latent noise is kept unchanged, the model still does not guarantee temporal consistency without extra training, but maybe you can have a try.

bananajoe182 commented 2 years ago

@zsyzzsoft Thank you very much, I will give it a try.