search

gordicaleksa / pytorch-neural-style-transfer

Reconstruction of the original paper on neural style transfer (Gatys et al.). I've additionally included reconstruction scripts which allow you to reconstruct only the content or the style of the image - for better understanding of how NST works.
https://youtube.com/c/TheAIEpiphany
MIT License
371 stars 78 forks source link

Why are there better results when using images in range [0, 255] instead of [0, 1]? #1

Open Nick-Morgan opened 4 years ago

Nick-Morgan commented 4 years ago

I was running into issues trying to re-create the original paper, and stumbled upon this repository.

I was able to re-create the results when using the caffe pretrained model (which has images in the range of [0, 255]), but had drastically different results when using pytorch's pretrained model (which has images in the range of [0, 1]). I noticed this tidbit of code in your repository:

https://github.com/gordicaleksa/pytorch-neural-style-transfer/blob/f5650deaf9c9aeac4afe31d2afde0a12326a776e/utils/utils.py#L43-L49

I applied that same transformation, and got results that are comparable to the original paper. I am somewhat confused about why this works, though. If pytorch's vgg19 is trained on millions of images in the range of [0, 1], wouldn't it just interpret anything above 1 as being pure white?

gordicaleksa commented 4 years ago

Hi Nick!

I have it on my backlog to try and make it work on the [0, 1] range as it feels more natural for PyTorch models as they were pre-trained, as you already said, on [0, 1] range imagery in contrast with those old caffe models.

What I did, because I was puzzled the same as you are, was pass in a [0, 255] say dog image into VGG and check whether the classification output is correct. And it was. The argument being (my hypothesis) that of symmetry. That's why it's working. VGG is able to do correct classifications even for [0, 255] range.

It should/must work for [0, 1] range I'd just need a bit more experimentation. If you figure it out before me please feel free to create a PR and notify me.