manuelruder / artistic-videos

Torch implementation for the paper "Artistic style transfer for videos"
Other
1.75k stars 252 forks source link

Processing stops with "bad argument #2 to 'add' (sizes do not match)" after first sucessful image #33

Closed HandsomeDevilv112 closed 7 years ago

HandsomeDevilv112 commented 7 years ago

Apologies if I've missed something very basic, but I'm a bit out of my experience and I don't know where to go from here. For a brief bit of background, I'm on a 1080, I can make neural style images in about a minute and a half without difficulty, I can do deep dream video with deepdreamanim, and I've been able to run the provided example. (Though I should note that it doesn't reach full iterations on each step) But, despite my best efforts, I come back to "bad argument # 2 to 'add' (sizes do not match)" after the first image is created. I appreciate all the help I can get, and don't be afraid of breakin' it down eli5 style. Thank you

al@al-teletran1:~/artistic-videos$ th artistic_video.lua -style_image /home/al/artistic-videos/example/1.jpg -content_pattern /home/al/artistic-videos/frames/frame_%04d.ppm -num_images 0 -num_iterations 900

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message.  If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192
Successfully loaded models/VGG_ILSVRC_19_layers.caffemodel
conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv3_4: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv4_4: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
conv5_4: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
Style image size: 486 x 631 
Setting up temporal consistency.    
Setting up style layer      2   :   relu1_1 
Setting up style layer      7   :   relu2_1 
Setting up style layer      12  :   relu3_1 
Setting up style layer      21  :   relu4_1 
Setting up content layer    23  :   relu4_2 
Setting up style layer      30  :   relu5_1 
Detected 5150 content images.   
Running optimization with L-BFGS    
<optim.lbfgs>   creating recyclable direction/step/history buffers  
Iteration 100 / 900 
  Content 1 loss: 5079415.000000    
  Style 1 loss: 25260.379028    
  Style 2 loss: 273015.502930   
  Style 3 loss: 165728.356934   
  Style 4 loss: 586837.890625   
  Style 5 loss: 2422.522545 
  Total loss: 6132679.652061    

SKIPPING A FEW

Iteration 900 / 900 
  Content 1 loss: 4669215.312500    
  Style 1 loss: 441.764927  
  Style 2 loss: 15081.632996    
  Style 3 loss: 27847.726440    
  Style 4 loss: 239924.487305   
  Style 5 loss: 2429.140282 
  Total loss: 4954940.064449    
<optim.lbfgs>   reached max number of iterations    
Running time: 280s  
Iteration 900 / 900 
  Content 1 loss: 4669215.312500    
  Style 1 loss: 441.764927  
  Style 2 loss: 15081.632996    
  Style 3 loss: 27847.726440    
  Style 4 loss: 239924.487305   
  Style 5 loss: 2429.140282 
  Total loss: 4954940.064449    
Reading flow file "example/deepflow/backward_2_1.flo".  
Reading flowWeights file "example/deepflow/reliable_2_1.pgm".   
WARNING: Skipping content loss  
Reading flow file "example/deepflow/backward_2_1.flo".  
WARNING: Skipping content loss  
Running optimization with L-BFGS    
WARNING: Skipping content loss  
/home/al/torch/install/bin/luajit: /home/al/torch/install/share/lua/5.1/nn/Container.lua:67: 
In 30 module of nn.Sequential:
./artistic_video_core.lua:286: bad argument #2 to 'add' (sizes do not match at /tmp/luarocks_cutorch-scm-1-4077/cutorch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:216)
stack traceback:
    [C]: in function 'add'
    ./artistic_video_core.lua:286: in function 'updateGradInput'
    /home/al/torch/install/share/lua/5.1/nn/Module.lua:31: in function </home/al/torch/install/share/lua/5.1/nn/Module.lua:29>
    [C]: in function 'xpcall'
    /home/al/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
    /home/al/torch/install/share/lua/5.1/nn/Sequential.lua:84: in function 'backward'
    ./artistic_video_core.lua:93: in function 'opfunc'
    ./lbfgs.lua:68: in function 'optimize'
    ./artistic_video_core.lua:120: in function 'runOptimization'
    artistic_video.lua:262: in function 'main'
    artistic_video.lua:359: in main chunk
    [C]: in function 'dofile'
    ...e/al/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00405d50

WARNING: If you see a stack trace below, it doesn't point to the place where this error occurred. Please use only the one above.
stack traceback:
    [C]: in function 'error'
    /home/al/torch/install/share/lua/5.1/nn/Container.lua:67: in function 'rethrowErrors'
    /home/al/torch/install/share/lua/5.1/nn/Sequential.lua:84: in function 'backward'
    ./artistic_video_core.lua:93: in function 'opfunc'
    ./lbfgs.lua:68: in function 'optimize'
    ./artistic_video_core.lua:120: in function 'runOptimization'
    artistic_video.lua:262: in function 'main'
    artistic_video.lua:359: in main chunk
    [C]: in function 'dofile'
    ...e/al/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
    [C]: at 0x00405d50
manuelruder commented 7 years ago

You need to calculate the optical flow, then pass it via the arguments -flow_pattern and -flowWeight_pattern. See the setup instructions and stylizeVideo.sh for more details.

HandsomeDevilv112 commented 7 years ago

That's the one. I completely misunderstood that step. If you have com here after walking the same path as I, try to run makeoptflo, follow the pathway instructions. It wants where your frames are, where it should produce the optical flow data, and two spaces for numbers. Though the way the help text is laid out, it looks like it should be [# [#]], but I got it to work with 1 1 Thank you much. Closing with comment that the problem was located between the keyboard and chair.