search

crowsonkb / style_transfer

Data-parallel image stylization using Caffe.
MIT License
113 stars 14 forks source link

What does --aux-image stands for ? #24

Open AbsurdePhoton opened 4 years ago

AbsurdePhoton commented 4 years ago

Hello,

I am writing a GUI for style-transfer, and one option is a mystery to me : of what use is the auxiliary image ? I have tested a little, thinking it could be a mask, but no.

So what is it good for ? There's an option aux-weight going with it.

Thanks.

crowsonkb commented 4 years ago

The auxiliary image allows you to specify an image which the rendering process is "drawn back to" during iteration. Technically it imposes an L2 penalty on the difference between the auxiliary image and the current image. The optimizer will try to decrease this penalty and thus the output will tend to drift towards/be influenced by the aux image. The --aux-weight option specifies the strength of this effect.

The use I had in mind for it was for was—when generating style transferred videos, to encourage temporal consistency with the previous frame by specifying the previous output as the aux image. Another use would be to "draw back" the rendered image to the initial content image by specifying the same image for both content and aux. Thirdly, artistic effects may be obtained by specifying other kinds of images as aux images.

AbsurdePhoton commented 4 years ago

Hey, thanks for the explanation. I'll try to explore a little more this option now that I understand more what it is good for.