search

EugenHotaj / pytorch-generative

Easy generative modeling in PyTorch
MIT License
428 stars 68 forks source link

Using Multiple Style Images in Neural Style Transfer #23

Closed thistleknot closed 3 years ago

thistleknot commented 4 years ago

Thank you for this code. I was able to get it to run in https://github.com/sorokine/docker-colab-local

I came across neural-style-gui that uses neural-style-pt and one of the features I'm lamenting that neural-style-gui has that your method does not is the ability to load multiple styles.

Is there anyway to restore that feature in your method?

EugenHotaj commented 3 years ago

Hey @thistleknot,

Thanks for the question. This should be supported with the current implementation (though I admit it's not very user friendly*). Currently, the style_img is provided as an input of batch_size=1. To use k style images, you could instead provide an input with batch_size=k where the style images are all stacked together into one batch. E.g.

style_img_1 = ... # load first style image
style_img_2 = ... # load second style image
...
style_img_k = ... # load kth style image

style_img = toch.cat((style_img_1, style_img_2, ..., style_img_k), dim=0)

However, as I mentioned in #1, I'm a little skeptical this will produce very pleasing results, especially when k is large, unless the images are very similar. This also seems to be the case in neural-style-pt where one style seems to dominate the other styles. Using some GAN techniques might give you better results.

* We could make this more user friendly by accepting a list of style_imgs in run_style_transfer. Pull requests welcome :).

EugenHotaj commented 3 years ago

Hey @thistleknot,

I added some basic support in for multiple styles in ef50a9c68a41e92078489f495e9680e05a5a1699.

We could probably extend this further to use different loss weights and style layers for each style.

thistleknot commented 3 years ago

Thank you! I've been manually merging images using transparency layers. I find your GAN method the most ... idk how you say. Beautiful. Deep dream sucks, and the original implementations of neural-style don't detect human faces very well at all.

thistleknot commented 3 years ago

As to your pull request.

I've been tempted to read an academic paper on how to implement neural-styles, I find understanding this technology a little intimidating but the papers all seem relatively short. I guess I'll never know how hard it is to implement unless I actually try.