Problems using converted model

[uricohen]

Hello, I tried to use a model I trained on pytorch from matconvnet using your wonderful project, but something is not right with the results I get from the converted code. I suspect this is some normalization issue or a similar incopetability between 'native' matconvnet model and pytorch-converted models.

So I took on of your converted model (resnet50-pt-mcn from your page) and tried to use it, but got the same result. Any suggestions from your experience is welcome... Thanks in advance, Uri

This is my code: `

    % Load network file
    modelPath = [modelsDir, 'resnet50-pt-mcn.mat'];
    net = dagnn.DagNN.loadobj(load(modelPath));

    % Allow extracting values from intermediate layers
    % https://github.com/vlfeat/matconvnet/issues/58
    net.conserveMemory = 0;

    I0 = single(my_read_image(i)); % Image in the 0..255 range

    % PyTorch normalization
    I1 = imresize(I0, net.meta.normalization.imageSize(1:2));
    I1 = I1 / 255 ; % scale to (almost) [0,1]
    I1 = bsxfun(@minus, I1, reshape(net.meta.normalization.averageImage, [1 1 3]));
    I = bsxfun(@rdivide, I1, reshape(net.meta.normalization.imageStd, [1 1 3]));

    % Feed the samples in batch
    net.eval({net.vars(1).name, I});
    % Extract features
    assert(length(net.vars) == length(net.layers)+1);
    N_LAYERS = length(net.layers)+1;
    for l=1:N_LAYERS
            s = prod(layer_sizes{l+1});
            data = reshape(net.vars(l+1).value, [s, N_SAMPLES])';
            % ...further use data...
    end

`

[albanie]

Hi @uricohen, thanks for including the sample (v. helpful)!

One issue with the code above is that for models that use batch norm (like resnets), it's important to ensure that the network is in "test mode" when producing features (this is done in the cnn_train_dag script here). This is because batch norm are handled differently at training and test time.

So in your example above you would add

net.mode = 'test'

before evaluating the network.

As a sanity check, I re-ran the resnet50-pt-mcn benchmark on imagenet and was within 0.1% of the numbers reported here: http://www.robots.ox.ac.uk/~albanie/mcn-models.html, so I think the underlying model should be OK.

[uricohen]

From my initial check this solves the problem. Bless you!! Tell me if you happen to visit Jerusalem, I'll buy you a beer :)