manuelruder / artistic-videos

Torch implementation for the paper "Artistic style transfer for videos"
Other
1.75k stars 252 forks source link

Out of memory. use Adam? downsample? #47

Open linrio opened 7 years ago

linrio commented 7 years ago
  1. I want to transfer higher piexl videos like 1k, 2K even 4k, however the size of my server's VRAM is 8GB (NV GTX 1070). With cudnn, the maximized size of image I can take from video is 1080x720p. Similar problem has been attention in #1.
    1. jcjohson said in Memory Usage that: we can use Adam instead of L-BFGS. And how to used this method in here?
    2. Also in OpenCL usage with NIN Model :
      th neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -gpu 0 -backend clnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0 ,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 -image_size 512 -optimizer adam

      And #1 @manuelruder suggested that:

      th artistic_video.lua ... -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 ...

      Above methods might have effect.

    3. How to downsample the video to reduce the memory?
    4. I think the ability of above method is limited. How To reduce memory usage?
linrio commented 7 years ago

I have tried the NIN model jcjohnson, however, it runs slowly in 1080x720p video on GTX 1070 (8GB), 8 cores i7 CPU:

8 Intel(R) Core(TM) i7-6700 CPU @ 3.4GHz
linrio commented 7 years ago

NIN and Adam can reduce the usage of VRAM actually, however the result is disappointed. 👎

ProGamerGov commented 6 years ago

Adam just needs it's parameters adjusted to achieve good results: https://github.com/jcjohnson/neural-style/wiki/Fine-Tuning-The-Adam-Optimizer