Error: incorrect size: only supporting singleton expansion (size=1)

[ProGamerGov]

@martinbenson , I believe I have found an issue. When trying to run deepmatting_seg.lua, I get the following error:

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message.  If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192
Successfully loaded models/VGG_ILSVRC_19_layers.caffemodel
conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv3_4: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv4_4: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
conv5_4: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
Setting up style layer          2       :       relu1_1
Setting up style layer          7       :       relu2_1
Setting up style layer          12      :       relu3_1
Setting up style layer          21      :       relu4_1
Setting up content layer        23      :       relu4_2
Setting up style layer          30      :       relu5_1
Capturing content targets
nn.Sequential {
  [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> (11) -> (12) -> (13) -> (14) -> (15) -> (16) -> (17) -> (18) -> (19) -> (20) -> (21) -> (22) -> (23) -> (24) -> (25) -> (26) -> (27) -> (28) -> (29) -> (30) -> (31) -> (32) -> (33) -> (34) -> (35) -> (36) -> (37) -> output]
  (1): nn.TVLoss
  (2): cudnn.SpatialConvolution(3 -> 64, 3x3, 1,1, 1,1)
  (3): cudnn.ReLU
  (4): nn.StyleLoss
  (5): cudnn.SpatialConvolution(64 -> 64, 3x3, 1,1, 1,1)
  (6): cudnn.ReLU
  (7): cudnn.SpatialMaxPooling(2x2, 2,2)
  (8): cudnn.SpatialConvolution(64 -> 128, 3x3, 1,1, 1,1)
  (9): cudnn.ReLU
  (10): nn.StyleLoss
  (11): cudnn.SpatialConvolution(128 -> 128, 3x3, 1,1, 1,1)
  (12): cudnn.ReLU
  (13): cudnn.SpatialMaxPooling(2x2, 2,2)
  (14): cudnn.SpatialConvolution(128 -> 256, 3x3, 1,1, 1,1)
  (15): cudnn.ReLU
  (16): nn.StyleLoss
  (17): cudnn.SpatialConvolution(256 -> 256, 3x3, 1,1, 1,1)
  (18): cudnn.ReLU
  (19): cudnn.SpatialConvolution(256 -> 256, 3x3, 1,1, 1,1)
  (20): cudnn.ReLU
  (21): cudnn.SpatialConvolution(256 -> 256, 3x3, 1,1, 1,1)
  (22): cudnn.ReLU
  (23): cudnn.SpatialMaxPooling(2x2, 2,2)
  (24): cudnn.SpatialConvolution(256 -> 512, 3x3, 1,1, 1,1)
  (25): cudnn.ReLU
  (26): nn.StyleLoss
  (27): cudnn.SpatialConvolution(512 -> 512, 3x3, 1,1, 1,1)
  (28): cudnn.ReLU
  (29): nn.ContentLoss
  (30): cudnn.SpatialConvolution(512 -> 512, 3x3, 1,1, 1,1)
  (31): cudnn.ReLU
  (32): cudnn.SpatialConvolution(512 -> 512, 3x3, 1,1, 1,1)
  (33): cudnn.ReLU
  (34): cudnn.SpatialMaxPooling(2x2, 2,2)
  (35): cudnn.SpatialConvolution(512 -> 512, 3x3, 1,1, 1,1)
  (36): cudnn.ReLU
  (37): nn.StyleLoss
}
Capturing style target 1
/home/ubuntu/torch/install/bin/luajit: /home/ubuntu/torch/install/share/lua/5.1/nn/Container.lua:67:
In 4 module of nn.Sequential:
/home/ubuntu/torch/install/share/lua/5.1/torch/Tensor.lua:322: incorrect size: only supporting singleton expansion (size=1)
stack traceback:
        [C]: in function 'error'
        /home/ubuntu/torch/install/share/lua/5.1/torch/Tensor.lua:322: in function 'expandAs'
        deepmatting_seg.lua:715: in function <deepmatting_seg.lua:702>
        [C]: in function 'xpcall'
        /home/ubuntu/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
        /home/ubuntu/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
        deepmatting_seg.lua:274: in function 'main'
        deepmatting_seg.lua:856: in main chunk
        [C]: in function 'dofile'
        ...untu/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x00405d50

WARNING: If you see a stack trace below, it doesn't point to the place where this error occurred. Please use only the one above.
stack traceback:
        [C]: in function 'error'
        /home/ubuntu/torch/install/share/lua/5.1/nn/Container.lua:67: in function 'rethrowErrors'
        /home/ubuntu/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
        deepmatting_seg.lua:274: in function 'main'
        deepmatting_seg.lua:856: in main chunk
        [C]: in function 'dofile'
        ...untu/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x00405d50
ubuntu@ip-Address:~/deep-photo-styletransfer$

I used this command to run deepmatting_seg.lua:

th deepmatting_seg.lua -content_image content.png -content_seg content_mask2.png -style_image Great_Wave_off_Kanagawa2.jpg -style_seg Great_Wave_Full_Colored_4_full_size.png -save_iter 50 -print_iter 50 -backend cudnn -cudnn_autotune -seed 876

The issue appears to be related to line number 715.

[ProGamerGov]

I resized all of my input images to the same height and width, and now it seems to work. I am not sure which of the images the problem was with, but maybe the mask images are not resized correctly alongside any resizing of the content/style images?

[ProGamerGov]

@martinbenson I solved the issue and submitted a pull request for the fix. It turns out that the masks were not resized to match their associated input images in the Lua code. Your deep_photo.py Python script does the resizing automatically before the Lua script is used, which is why you did not come across this error that affects manual usage of the deepmatting_seg.lua script.

I have also created a standalone version of the deepmatting_seg.lua script without the photorealism features, for those who wish to use the regular Neural-Style with the new segmentation/Multi-Region Spatial Control features: https://gist.github.com/ProGamerGov/bcbd27a3d2e431adb73ef158d9990d93

[ProGamerGov]

I have been getting this error with the NIN model, and I am not sure why. My images are correctly sized, yet it still gives an error that appears to be related to the style images.

[ProGamerGov]

This post here indicates that NIN does work in the old version of the code: https://github.com/luanfujun/deep-photo-styletransfer/issues/29#issuecomment-290845078 I tested this and it does not seem to work.

I know that NIN works with the current version of Neural-Style as well, so there may be an issue somewhere in the mask code?

[ProGamerGov]

@martinbenson

This line of code is the cause of this issue:

local l_mask = l_mask_ori:repeatTensor(1,1,1):expandAs(input)

https://github.com/martinbenson/deep-photo-styletransfer/blob/master/deepmatting_seg.lua#L716

[wulabs]

I also received this error with default VGG-19 and image_size=512 when I used a non default style_scale. It ran fine with style_scale=1.0