search

gustavla / self-supervision

Training and evaluating self-supervised deep neural networks
BSD 3-Clause "New" or "Revised" License
29 stars 6 forks source link

Converting colorization-pretrained model to RGB #6

Open aravindhm opened 6 years ago

aravindhm commented 6 years ago

Hi Gustav,

I downloaded the pretrained VGG-16 model from http://people.cs.uchicago.edu/~larsson/color-proxy/models/vgg16.caffemodel.h5 and tried to fine tune it for pascal classification using the script in selfsup/evaluate/__main__.py . Unfortunately, the pretrained model's conv1_1 filters are meant for single channel grayscale inputs but the model defined by voc_classification.py is for colour images. This results in the following asserting failure 

(tensorflow3) aravindh@gnodeb1:~/projects/self-supervision$ CUDA_VISIBLE_DEVICES=0 python3 selfsup/evaluate/ voc2007-classification /users/aravindh/scratch/autocolorize/vgg16.caffemodel.h5 -n vgg16 --output /users/aravindh/scratch/self_supervision/gustavia/voc_vgg16/classification/ --limit 100 
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "selfsup/evaluate/__main__.py", line 26, in <module>
    main()
  File "selfsup/evaluate/__main__.py", line 23, in main
    time_limit=args.limit, iterations=args.iterations, network_type=args.network)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 476, in train_and_test
    train(*args, **kwargs)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 191, in train
    network_type=network_type)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 91, in build_network
    use_dropout=True)
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 362, in build_network
    z = conv(z, 64, name='conv1_1')
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 303, in conv
    return vgg_conv(z, num(ch), **kwargs)
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 164, in vgg_conv
    assert W_shape is None or tuple(W_shape) == tuple(shape), "Incorrect weights shape for {} (file: {}, spec: {})".format(name, W_shape, shape)
AssertionError: Incorrect weights shape for conv1_1 (file: (3, 3, 1, 64), spec: [3, 3, 3, 64])

In order to replicate the results in column 1 of table 1 in your paper (http://arxiv.org/pdf/1703.04044.pdf), please let me know what should be changed.

Best wishes, Aravindh Mahendran

aravindhm commented 6 years ago

The code runs if I add a x = tf.reduce_mean(x, axis=3, keep_dims=True)

at line 171 in voc_classification.py ... I am concerned whether the mean subtraction needs to change as well.