Training Error, happens with any input style image

[backnotprop]

• Cuda 9
• python 2.7

Traceback (most recent call last):
  File "main.py", line 287, in <module>
    main()
  File "main.py", line 40, in main
    train(args)
  File "main.py", line 136, in train
    utils.init_vgg16(args.vgg_model_dir)
  File "/home/ubuntu/train/experiments/utils.py", line 100, in init_vgg16
    vgglua = load_lua(os.path.join(model_folder, 'vgg16.t7'))
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 606, in load_lua
    return reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 242, in read_nn_class
    attributes = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 317, in wrapper
    obj = build_fn(reader, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 242, in read_nn_class
    attributes = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 593, in read
    return self.read_table()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 570, in read_table
    v = self.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 146, in read_tensor
    storage = reader.read()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 591, in read
    return self.read_object()
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 521, in wrapper
    result = fn(self, *args, **kwargs)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 544, in read_object
    return reader_registry[cls_name](self, version)
  File "/home/ubuntu/train/experiments/train/local/lib/python2.7/site-packages/torch/utils/serialization/read_lua_file.py", line 165, in read_storage
    return python_class.from_buffer(reader.f.read(size), 'native')
ValueError: buffer size (53728077) must be a multiple of element size (4)

[backnotprop]

I've reverted to Cuda 8 and I get the same issue

[zhanghang1989]

This looks like a bug with PyTorch. I will convert the vgg weights later to avoid this issue.

[backnotprop]

@zhanghang1989 yea I figured, I submitted an issue to PyTorch as well.

Is there an alternative I can use for now?

[backnotprop]

pytorch issue here

[backnotprop]

I believe my download must've been corrupted ... Ive downloaded a separate vgg file and it seems to be working now... waiting to see how training goes

[zhanghang1989]

Great. please keep me posted.

[ss32]

@backnotprop what directory structure does the training expect? And are labels generated from the directory names or individual images?