Euclidean Loss

[dasguptar]

Hi, I was wondering what changes would I need to make to incorporate an Euclidean loss function in place of the standard log loss function. Would simply passing the outputs of the softmax layer and calculating the difference suffice? If I use this as the final layer, do I need to compute the derivative?

[bazilas]

you should mainly create a new loss function and include it at vl_simplenn. You should also change the error estimation at cnn_train.

I am currently using the library for 2D human pose estimation and it works really nice.

[zoharbarzelay]

Hi Bazilas,

Do you mind adding a simple example for your 2d loss function? Something is not working for me. I've used the follwoing code for a squred loss fully-connected layer: function Y = vl_nnL2(X,c,dzdy) if nargin <= 2 Y = 0.5*sum((squeeze(X)'-c).^2);
else Y = -((squeeze(dzdy)'-c)); Y = reshape(Y,size(X)); end

and then my last layer is defined as: net.layers{end+1} = struct('type', 'nnL2') ;

[bazilas]

You can minimize the squared difference between the ground-truth(Y) and the predictions (X):

E = (Y-X)^2.

Then in the backward step you will have:

dE/dX= -2*(Y-X).

You can omit 2, because the learning-rate should compensate for it.

In my case, it worked very nice for some datasets (human pose estimation).

cheers, Vasilis

[zoharbarzelay]

Vasilis, so just as a sanity check, I would appreciate your help:

to run a simple one-layer 5-d linear regression, I would define the following inputs: N = 10000; x = rand(5,N); weights = [1,2,3,4,5]'; y = weights'_x + 0.3; n = randn(1,N)_0.01; y = y + n; %figure,plot(x,y,'.');

n = 0.8_N; imdb.images.data = single(reshape(x,1,1,1,[])) ; imdb.images.labels = y(:)' ; imdb.images.set = [ones(1,n) 3_ones(1,N-n)] ; imdb.meta.sets = {'train', 'val', 'test'} ; imdb.images.data = bsxfun(@minus, imdb.images.data, mean(imdb.images.data,4)) ;

the following net: net.layers = {} ; net.layers{end+1} = struct('type', 'conv', ... 'filters', single(f_randn(1,1,1,5)), ... 'biases', single(zeros(1, 5)), ... 'stride', 1, ... 'pad', 0) ;
net.layers{end+1} = struct('type', 'conv', ... 'filters', single(f_randn(1,1,5,1)), ... 'biases', single(zeros(1, 1)), ... 'stride', 1, ... 'pad', 0) ;
net.layers{end+1} = struct('type', 'nnL2') ;

and then have my nnl2 function like this: function Y = vl_nnL2(X,c,dzdy) if nargin <= 2 Y = 0.5*sum((squeeze(X)'-c).^2);
else Y = +((squeeze(dzdy)'-c)); Y = reshape(Y,size(X)); end

[bazilas]

In the backward step:

Y = +((squeeze(X)'-c))*dzdy; Y = reshape(Y,size(X));

[yangcong955]

@bazilas Hi Bazilas, I would appreciate if you could give me a help. I would like to add my own loss layer into a DagNN network, but I don't know how to do it. I found a guidance in https://github.com/vlfeat/matconvnet/issues/396, but it is still not clear. It would be perfect if you can use the example in this page.

[h612]

Isn't X a probability and c a class of int, in this case for classes 1-10? How can we do (squeeze(X)'-c) Also does this consider a batch of 100 images?