Using NIN Imagenet model

[vidyapriyap]

Hi, I was trying to work with the NIN imagenet model. I was not able to to produce good results with the parameters I experimented with. Could you let me know the content and style layers used and the parameters which you set for any example?

[htoyryla]

vidpri13 notifications@github.com kirjoitti 12.5.2016 kello 4.06:

Hi, I was trying to work with the NIN imagenet model. I was not able to to produce good results with the parameters I experimented with. Could you let me know the content and style layers used and the parameters which you set for any example?

My 1920px examples here http://liipetti.net/erratic/tests/1920px-tests/ use nin-imagenet-conv and you’ll find all parameters for each image.

I would no longer, however, recommend using relu12 for reasons I have given here https://github.com/jcjohnson/neural-style/issues/217

Another question then is, what counts as good quality for each of us, which again might depend on what kind of images and styles we work with. I guess nin-imagenet-conv is not so good for crystal sharp or photorealistic results.

[ProGamerGov]

@vidpri13 @htoyryla

Try using:

th neural_style.lua -tv_weight 0.0001 -style_image style.jpg -content_image content.jpg -output_image out.png -image_size 1200 -save_iter 50 -content_weight 10 -style_weight 1000 -num_iterations 1000 -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -content_layers relu0,relu1,relu3,relu5,relu7,relu9,relu12 -style_layers relu0,relu1,relu3,relu5,relu7,relu9,relu12

Or:

th neural_style.lua -style_image style.jpg -content_image content.jpg -output_image out.png -image_size 1200 -content_weight 10 -style_weight 1000 -save_iter 50 -num_iterations 1000 -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -content_layers relu0,relu1,relu3,relu5,relu7,relu9,relu12 -style_layers relu0,relu1,relu3,relu5,relu7,relu9,relu12

Setting the -tv_weight to 0.0001 will make the image really sharp. Setting -content_weight to 10 and -style_weight to 1000 will help produce good results with many styles, but you should play around with the weights. The -style_layers and -content_layers are some of the most important parts of creating images. I find good results from using them all, but you can play around with omitting some layers.

I also would recommend using -save_iter 50 or -save_iter 10 combined with -num_iterations 400. Some of the best results will occur between 50 and 200 iterations, but occasionally some of the best results will be a bit before or after that range of iterations.

[htoyryla]

Like I mentioned above, relu12 (and other topmost layers) might not be a good choice, at least not for a content layer.

While VGG19 consists of

• convolutional layers which recognize features (shapes, textures, colors) in images
• FC layers which classify images ("there is a sofa and a dog ")

NIN is different, though, in that it has no FC layers. Convolutional layers handle everything, even the classification. What this means is that the top conv layers in NIN are far more abstract than in a VGG. You can reconstruct a recognizable image of a dog on a sofa from conv5_4 in VGG19, but relu12 in NIN just knows that there is probably a dog and maybe a sofa in the image (but not how they look like).

In neural-style terms, letting VGG19/conv5_4 decide the content, the result can look like this (VGG19 with defaults except conv5_4 for content, at 550 iterations).

while a similar run with NIN/relu12 resulted in this (content weight increased to 2000 to get a similar balance with style, image is after 400 iterations. Shortly afterwards the optimization ran wild and the losses exploded).

This means that relu12 hardly contributes to image quality unless one is after that kind of effect. The same applies to relu11, most probably to even lower layers than that. I have not tested which would be the highest usable layer (out of which an image somehow resembling the original can reconstructed by neural-style), but doing so should be simple.

PS. In practice it appears that including relu12 doesn't do much harm either. This is probably because the loss values obtained from it are so small compared to the lower layers.

[htoyryla]

"Setting the -tv_weight to 0.0001 will make the image really sharp."

Thanks for this tip. Somehow I had not realized that, with NIN, it would be possible to get sharper results with a lower tv_weight without getting noise. Need to experiment with this.

[dovanchan]

@htoyryla Hi, this discussion was in last year, until now,How about your experiment. Can you share some tips about the parameters in neural style transfer ,I'm looking forward to your share。

[dovanchan]

@ProGamerGov Hi, this discussion was in last year, until now,How about your experiment. Can you share some tips about the parameters in neural style transfer ,I'm looking forward to your share。

[sdziscool]

@dovanchan look in the wiki to find ways to improve your results. Each image needs a different approach, but in the wiki the rough outlines for what each parameter does are defined. Also more neural models can be found with their instructions.

[dovanchan]

Hi,the first algorithm is from Dreamscope the second one is my result; It's obviously that dreamscope can control the result without effecting face;Anyone knows how dreamscope do that?What parameters should we adjust for doing that?

[htoyryla]

There's no simple tweaking of parameters in neural-style to do that.

In principle the process is simple: first segment the image into the foreground (person) and background, then use masks to process both with different styles. We have discussed the second part recently here. The first part is more tricky... it can be done automatically with a properly trained feed-forward network, but where to find one that would work well enough? I guess those who invest a lot of time and effort into training a good segmentation solution also put a price tag on their product.

[dovanchan]

@htoyryla --content_layers, default='1_1,2_2,3_3,4_4,5_4' -style_layers, default='2_2,5_4' -content_weight', default=10 -style_weight', default=200,

I used this parameter,But it always give me a bad result; Do you have some recommend for using VGG-19 model?Can I refer your parameters?

[htoyryla]

@dovanchan I am not sure if I understand your question. If you are using VGG19 your layers should have names like relu2_2. If you don't know where to start, just use the defaults. Your settings above are not the defaults, they are relu4_2 for content and relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 for style.

[dovanchan]

@htoyryla what I mean is that: My content layers using:relu1_1,relu2_2,relu3_3,relu4_4,relu5_5 style layers using:relu2_2,relu5_4; But the result is strange,So I wanna refer your parameters that help me get a great result what I want

[htoyryla]

Can't see why these settings would give a strange result. You might try without relu5 layers, though. And there is no relu5_5 in vgg19.

As to my parameters, I don't have any. Each time I have to find the settings that work for what I am trying to achieve.

This is what I get using your parameters and otherwise default settings.

[dovanchan]

@htoyryla The first one is my result ,The second one is result use dreamscope. The difference is far away，Are there any thought that can let me get close to it;

[htoyryla]

Unfortunately I cannot help. My research is not targeted to imitating what the style transfer apps and services are doing, but rather the opposite.

[dovanchan]

I tried to modify the parameter,but the image is so abstract. If I wanna increase the size of the texture and make color more bright-coloured what parameters can I modify?

[htoyryla]

If the style texture is too small, you have to make a smaller image and then enlarge it. There is even a script for this, see the following issue thread https://github.com/jcjohnson/neural-style/issues/351

Using original-colors option may help for the color.

[minxdragon]

Have you specified -init image? Because that made an enormous difference to my results.

On Fri, 7 Apr 2017 at 7:37 pm, Hannu Töyrylä notifications@github.com wrote:

If the style texture is too small, you have to make a smaller image and then enlarge it. There is even a script for this somewhere here.

Using original-colors option may help for the color.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jcjohnson/neural-style/issues/226#issuecomment-292490140, or mute the thread https://github.com/notifications/unsubscribe-auth/ARL-aXQxMFdHgEeNs9UO-ui8nZLowD5Tks5rtgPpgaJpZM4IcoO1 .