search

titu1994 / Neural-Style-Transfer

Keras Implementation of Neural Style Transfer from the paper "A Neural Algorithm of Artistic Style" (http://arxiv.org/abs/1508.06576) in Keras 2.0+
Apache License 2.0
2.26k stars 481 forks source link

Resulting style does not really match source #4

Closed bmaltais closed 8 years ago

bmaltais commented 8 years ago

I am running into a strange issue with the code. The output in some case never "converge" to look like the source style. Here is an example:

https://goo.gl/photos/Vf76NHAa4Pj5Nyw87

See how the result after 50 epoch is still far from looking like the source? Way too much green and dark brue areas. It is possible something is wrong? I am using the following parameters to generate she output:

python INetwork.py ../in/t5/dst.png ../in/t5/src4.jpg ./res4b --image_size "476" --content_weight 0.01 --num_iter 50 --min_improvement 0.05

bmaltais commented 8 years ago

I have added a 2nd example. It is almost as if it does not fully apply the source style color to the content image. Even with content weight of 0.

titu1994 commented 8 years ago

@bmaltais Content weight is a parameter used to define how much of the structure of the content should be preserved. In this case, what you should have tried instead is increasing the style weight instead, which defines how strongly the style will overpower the content.

An example where results similar to what you desire : woman_at_iteration_50

The parameters for this result are :

python inetwork.py "D:\Yue\Pictures\content.png" "D:\Yue\Pictures\Art\candy-style.jpg" "D:\Yue\Desktop\Neural Art\Woman" --style_weight 1000 --num_iter 50

The obvious drawback of such a high style weight (1000) is the fact that the woman's facial structure and color are drastically altered. Look at the lip region when compared to the original, as well as the eyes (blue color of her iris has become yellowish).

Such artifacts can be removed with MRF regularization, however since it is almost a brute force patch matching technique, execution speed drastically increases. So preservation of content structure comes at the heavy cost of execution time.

bmaltais commented 8 years ago

I see. I wrongly presumed that style weight to content weight of 1 : 0 would give the same result as, say, 1000 : 0 but apparently the algorythm is not totally taking the 0 weight for content as 0 and applying full on style to the init image no matter what the style weight is.

I will dome more experimentation then.