search

crowsonkb / style_transfer

Data-parallel image stylization using Caffe.
MIT License
113 stars 14 forks source link

More of a question #3

Closed 1N50MN14 closed 8 years ago

1N50MN14 commented 8 years ago

First of all thank you for the awesome work and apologies if my question sounds naive/silly as I'm a newbie.

I went through README (twice) and my understanding is that following the installation instructions style_transfer.py <content_image> <style_image> would yield a content_image styled accordingly. So you're basically both training as well as applying the style at the same time and if so isn't that a very slow process comparing to applying a pre-trained model (even better using using feed forward network)?

crowsonkb commented 8 years ago

As in jcjohnson's new paper? It took them four hours to train a neural network to apply one particular style but after training they could apply it hundreds of times faster than the technique of Gatys et al., which just optimizes an image with regard to style and content loss functions, which this is an implementation of. This technique's performance depends on the image resolution, but it typically takes minutes rather than hours or seconds.

1N50MN14 commented 8 years ago

Yes you're correct I was referring to jcjohnson's new paper; very interesting the Golden Gate Bridge + Starry Night style transfer you're generating is the most astonishing / accurate I've seen using the new technique, very impressive!

From a practical point view would it be possible for the python script to output the trained model so that it could be reused as an alternative source input for content_image to avoid training the model for a sequence of images sharing the same style?

My use case is that I have a 15 seconds videos (30fps) which I'm decomposing to 450 400x800px frames then applying a pre-trained model on each video in parallel over X number of machines, and finally merging them back into a single video.

1N50MN14 commented 8 years ago

Sorry forgot to mention, this is the implementation I'm currently using https://github.com/yusuketomoto/chainer-fast-neuralstyle, it's very fast as you can see in the repo's README (seconds) but is often a hit and miss with output result.

crowsonkb commented 8 years ago

This is still the old technique though, so there isn't a trained model to output. It just takes several minutes per frame.

1N50MN14 commented 8 years ago

Ah ok got it, thank you for the clarification (closing open issue)