All brushes same size

[Heartsie]

Hi! It looks like there is a problem where all the "brush strokes" in a style are all the same size. You can really see it here: This was the style image I used:

40,000 iterations of the microsoft coco dataset. styleweight of 3, stylesize of 512, image size of 512, learning set to .001, batch of 1. I got the same results with different style weights and learning sizes. Unfortunately this looks like a big limitation :(

[Heartsie]

How is it possible to get something like this image where you can not see a visible "brush"?

[luweishuang]

what style layers and content layers you have used, i used the default layers and your parameters and cann't get transformed image as your with 5000 iters in COCO dataset, does iterations affect the final result?

[Heartsie]

You can see the style image I used. These were my settings: 40,000 iterations of the microsoft coco dataset. styleweight of 3, stylesize of 512, content weight of 1, normalize set to true, image size of 512, learning set to .001, batch of 1.

But I want to know how to make a image that looks like the example included with the project! When I try to make a image like that I get results like this:

But sadly I don't think it is possible.

[bmaltais]

I have a feeling even Dmitry would not be able to produce a good model with the latest code. His previous example were from a totally different code base. Perhaps this new code base is not producing results that are as good as the one before.

Again, if Dmitry provided the actual command line he used to produce a good model it would help the community learn what is a good base to start from instead of going through major trial and error trying to figure out a good reference set of commands that produce decent result.

Personally I have given up since I don't have a GPU fast enough to try to find something that work. Takes way to long to get something that look just like what you posted ;-(

What would be nice would be to have the ability to use, say, commands from neural-style that you know work well and use those using Dmitry to produce a good model from them...

[Heartsie]

I think I agree bmaltais. Is there a way to get the code used to make the good examples?

[sheerun]

cannot reproduce as well

[xpeng]

is there any parameter like "style scale" mentioned by neural-sytle. i agree bmaltais too, does any one tried code below tag texture_nets_v1

[DmitryUlyanov]

I will take a look today. Probably I cut something important..

On 10 Aug 2016 10:39 a.m., "XPeng" notifications@github.com wrote:

is there any parameter like "style scale" mentioned by neural-sytle. i agree bmaltais too, does any one tried code below tag texture_nets_v1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/texture_nets/issues/28#issuecomment-238789552, or mute the thread https://github.com/notifications/unsubscribe-auth/AGanZDum714dXBLQgcsn3IJ_lt1hw2R8ks5qeYBBgaJpZM4JfIYT .

[Heartsie]

Thanks Dmitry! It would also be awesome to have the training settings for just one model and a example of that model. It would go a long way.

[bmaltais]

@Heartsie Regarding your original question, I am curious about the following:

What was the actual size of the style image? Was it 512 x "less or equal to 512"? What was the size of the content image when you tried to apply the style? Was it 512 x "less or equal to 512"?

I have a feeling that unless a style scale option is added to test.lua style size will remain fixed at 512 x something... so if your content size was 1024x1024 the brush strokes will appear small. An option to apply a scaling factor of 2 would be needed to scale them up so they are proportional for 1024x1024 images...

So make sure your style image is scaled at 512 x "512 or less" before training. If not then my understanding is that train will re scale it down to 512 and the "brush strokes" will appear smaller than you expected.

Next, make sure your content image you want to apply the style on is also scaled down to 512 x 512 or smaller. This will ensure that "brush strokes" will remain consistent with the style sizing. You will then need to upscale the produced output to HD so it look good on a large screen/print.

[Heartsie]

Actual size of the style image is 512, the content image was 512 as well. The main problem isn't as much the size of the brush, but the fact that you can see a very visible grid-like pattern and all the "brushes" are prety much the same size. Unlike swirly picture where it looks much more organic. I can live with the fact that the brush size is built into the model but it is the lack of organic feel that gets me.

although if would be nice to be able to scale the feature size :)

[bmaltais]

@Heartsie I totally agree then. Something is wrong as the produced style is not looking at all like the source... I am not sure about the grid pattern. From experience with neural-style adjusting the tv-weight to 0.000085 gave me really smooth yet sharp results. Not sure if this tv-scale apply to this solution as is... But the current train.lua does not allow to set the tv-weight

What would be great is if train.lua was actually based on neural-style from a parameter/training point of view. It would allow one to find a good set of working arguments using the standard neural-style solution and then apply those to create a resulting static model for a given style input image... not sure if it would be doable...

[bmaltais]

I also noticed the current training code use adam optimizer instead of the more commonly used lbfgs. I have a feeling the current training code is just not really good compared to the previous one...

[Heartsie]

Its so fast though! Hopefully Dmitry can figure out these issues and keep the insane training speed.

On Wed, Aug 10, 2016 at 9:36 AM, bmaltais notifications@github.com wrote:

I also noticed the current training code use adam optimizer instead of the more commonly used lbfgs. I have a feeling they current training code is just not really good compared to the previous one...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/texture_nets/issues/28#issuecomment-238868291, or mute the thread https://github.com/notifications/unsubscribe-auth/AJqpIhklO2XqHEz6IthE1vfKYjfAgrfcks5qedPMgaJpZM4JfIYT .

-=Robin Graham Postrgram.com

[bmaltais]

I agree about speed but not at the cost of significant quality loss. In this current code release I would say it is not worth since it does not appear to be possible to produce good models.

[bmaltais]

@Heartsie What GPU do you use and how long does it take you to train a model by the way? What dataset do you use?

[Heartsie]

I have a 1070. I use the coco dataset and it takes maybe 4 hours to go through 50,000 images. I capped training at 50,000 images because it didn't really change the model much past 40,000.

On Wed, Aug 10, 2016 at 9:50 AM, bmaltais notifications@github.com wrote:

@Heartsie https://github.com/Heartsie What GPU do you use and how long does it take you to train a model by the way? What dataset do you use?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/texture_nets/issues/28#issuecomment-238872316, or mute the thread https://github.com/notifications/unsubscribe-auth/AJqpIi_ARHAMzFwU6b5GtkSc1bdvS9-pks5qedcVgaJpZM4JfIYT .

-=Robin Graham Postrgram.com

[bmaltais]

Nice to know. I am debating whether on not to by a new rig for deep learning. I currently run models on my old iMac 27 with an nvidia GPU... way to slow for texture_net at the moment. But 4 hours on a 1070 is looking good. I wonder how much faster a 1080 would be.

[Heartsie]

And texture net is pretty fast too! The same training takes like 20 hours on chainer-fast-forward

On Wed, Aug 10, 2016 at 10:57 AM, bmaltais notifications@github.com wrote:

Nice to know. I am debating whether on not to by a new rig for deep learning. I currently run models on my old iMac 27 with an nvidia GPU... way to slow to texture_net at the moment.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/texture_nets/issues/28#issuecomment-238893562, or mute the thread https://github.com/notifications/unsubscribe-auth/AJqpIlguxOXaSJK12alZ7pM5U7EV6Z9Xks5qeebSgaJpZM4JfIYT .

-=Robin Graham Postrgram.com

[rayset]

I'm helping op. The style parameters were likely set 512 ,and so was image size. There is no scaling factor in the parameters as far I know, how do you choose "512 or less"?

[xpeng]

@DmitryUlyanov Thanks! so far in my test of some method, texture_nets is faster than others, so hope to be front runner while keep quality.

[randomrandom]

@Heartsie normalizing gradients requires a whole new hyperparameter tuning ;) try without it

[DmitryUlyanov]

@Heartsie @xpeng @sheerun Hello, first you need to understand, that you need to explore style images which work good. I've tried hundreds of images, with tons of different parameters to obtain results from the paper. I tried my code and it works. Generally you want your stly image to be flat and contain rather huge details (your brush example bad in this way). This style from the paper Img works good for me, you should select styles like that. I'm also not sure why your first image is such desaturated, I bet @182f5413b99246106f205718643e91ae25aa7f36 will fix it. The image from you second post is something you normally get with wrong parameters. You need to tune them. I've sometimes used the trick when you gradually change the style weight as you optimize, you can code it, it's easy. Justin used Tanh for his model, you can try too, I did not notice too much difference.

I cannot disclose the exact parameters for now, I am sorry. If you are writing a paper, I can train a model with a style you need and send you examples.

I've also added TVLoss, you can use it to get rid of noise to some extent.

@bmaltais feed-forward neural style never used L-BFGS as we perform stochastic optimization

[Heartsie]

Thanks @DmitryUlyanov ! What are the main parameters you experiment with that have the greatest impact on a good looking model?

[htoyryla]

I just got quite good results from a Matisse cutout as a style, training with MSCOCO images for a couple of hours on a 1070. Here's an example result image. Not my favorite style, exactly, but technically I am satisfied :) Experiments with other styles have not yet given good results, probably the style images were too complex.

[rayset]

That model is nice! What parameters did you use?

[htoyryla]

I am not fully sure about the parameter values that finally worked. Image and style size probably 512, style_weight even if I experimented with higher values I probably at the end had it back at 1. Johnson model in use. Probably the most important thing is that the style image was very clear and simple: very clearly two-dimensional structure and large shapes in blue and white.

The funny thing is that the style in the output picture (above) is actually quite different from the original style image. The original was sparse and simple, the output is much more detailed, but it works. So probably the trick is finding style images that produce good results, rather than expecting the results to automatically copy the original style as such.

Also, I found the display very helpful. Testing different style images, one can usually see quite soon if it will not work at all. If it looks promising, then letting it run further seems to add color and quality, up to some point.

[DmitryUlyanov]

@htoyryla Nice result. I think it's all about style image too, I spent several days tuning params for a certain style and never got good results. For others any parameters worked very well.

[zikzxak]

I have to give it to you, the image below is absolutely beautiful. I've tested a lot of different combinations (and will keep testing) but haven't gotten near your results. I have some good results but they are usually really close to the content image colorwise. You were able to insert lots of style (color, shapes) while keeping the figure of the girl. Feels quite impossible considering my numerous tests but maybe there's something I'm missing. I'm playing with style_weight, content_weight, style_layers

Mine:

[DmitryUlyanov]

@zikzxak Since only ratio between content_weight and style_weight matters, you can fix content_weight to 1 and play only with style weight. Your image clearly has less style than mine.

Try the other parameters, mentioned here https://github.com/DmitryUlyanov/texture_nets#training. And play with test image_size.

[zikzxak]

@DmitryUlyanov Thanks for the tip! I just wrote a quick script to play with SW so let's see. Exciting :)

[rogozin70]

Anyone can upload on a forum t7 models? Let's create a collection filters?

[zikzxak]

I had a small revelation when I actually studied how the system works and what each parameter does. Now getting similar results but will train more to confirm. Thanks @DmitryUlyanov !

[xpeng]

@zikzxak have you got the similar results now? is it just train more or use the different parameters exactly?

i meet the same problem, i want to train with this style:

and i got following result after 34000 iters using normal parameters: -model johnson -image_size 512 -style_size 512 -content_weight 1 -style_weight 50 -learning_rate 0.001 -normalize_gradients true -tv_weight 0.000085

i tried different image_size or style_weight, but things are not changed much. seems training can not capture some large features.

by comparison, i use neural-style with a result well enough(1000 iters):

[htoyryla]

As far as I can see, the size parameters image_size and style_size matter very much as to the scale of the stylistic features. Haven't checked the code yet, but looking at the display, it appears that actual image used for training is a crop of size image_size taken from the original training image. Thus, the size of the original training image is also relevant, if one is using own's own images. A 512x512 crop from a 2000x2000 image is totally different from the same crop from a 720x720 image.

[zikzxak]

I got similar results colorwise and understood how that works but i'm still working on the ratio between original vs style. I'll post more info when I get good results and figure it out.

[zikzxak]

Getting close. It is hard to balance between not blending the colors but still blending the shapes/styles. I'll do some more testing.

[sheerun]

@zikzxak How long did you train on what GPU for such result?

[lvjadey]

@zikzxak what is the parameters?

[zikzxak]

@sheerun I trained with 512 images with Titan X Pascal. Didn't time it but about 2 hours for 50k iterations. I was thinking doing more iterations to see if the colours would be more clear.

[rogozin70]

2.5 hours - gtx 1070, style weight-10

[kaca0083]

@DmitryUlyanov From https://github.com/DmitryUlyanov/texture_nets/blob/master/src/texture_loss.lua I see that the gradient normalization part is commented, are you sure -normalize_gradients true works as supposed?

[DmitryUlyanov]

@kaca0083 Thanks for reporting, originally I wanted to get rid of that flag. Removed with https://github.com/DmitryUlyanov/texture_nets/commit/1c804fdce883b6ce40d1ca1eda558bcaab64afd0

[kaca0083]

@DmitryUlyanov So you mean gradient normalization is unnecessary?

[DmitryUlyanov]

@kaca0083 Well, I did not observe much difference with and without. It just needs different style_weight setting.

[lancodec]

@zikzxak I have tuned style_weight many times while content_weight is 1,but can not get the desired results like yours, what parameters i need to tune? tv_weight or image_size or other parameters?

[lancodec]

@DmitryUlyanov I have tuned style_weight many times while content_weight is 1,but can not get the desired results like yours, what parameters i need to tune? tv_weight or image_size or other parameters?

[rogozin70]

2.5 hours - gtx 1070, style weight-10

[rayset]

This One is Pretty good. Mind sharing the style image? Also, what image size was it ? 512? Any other non default parameters?

[rogozin70]

512 size image, style size 512 style weight 200 content weight 10