matt-weinstein / adigator

Matlab Algorithmic Differentiation Toolbox
25 stars 5 forks source link

Incorrect gradient values when evaluating the following function #16

Open daybrown opened 2 years ago

daybrown commented 2 years ago
function y = nn_test(x,L)

W1 = randn(2,2);    B1 = randn(2,1);
W2 = randn(2,1);    B2 = randn;
W = {W1,W2};    B = {B1,B2};
a = cell(1,2);
for i = 1:2
    if i == 1
        a{i} = W{i}'*x + B{i};
    else
        a{i} = W{i}'*a{i-1} + B{i};
    end
end
a{1} = W{1}'*x + B{1};
a{2} = W{2}'*a{2-1} + B{2};
y = a{end};

end

Another example

function y = nn_test(x,L)

n = length(L);
NN = L(1:n-1) * L(2:n)' + sum(L(2:n));
p = randn(NN,1);
np = length(L) - 1;
inds = 1;
a_b = x;
for i = 1:np
    nums = L(i)*L(i+1);
    w = reshape(p(inds:inds+nums-1), L(i), L(i+1));
    inds = inds+nums;
    nums = L(i+1);
    b = reshape(p(inds:inds+nums-1), L(i+1),1);
    inds = inds+nums;
    a = w'*a_b + b;
    a_b = a;
end
y = a; 

end

Both examples are two representations of a same function, the jacobian file is produced by

N = 2;
x = rand(N,1);
L = [2 2 1];

gx = adigatorCreateDerivInput([N, 1],'x'); % Create Deriv Input
genout = adigatorGenJacFile('nn_test',{gx,L});
S = genout.JacobianStructure;

[Jac,y] = nn_test_Jac(x,L);

For the first example, it returns a 2 by 2 Jacobian, but the function is R^2 to R^1, it should have 1 by 2 Jacobian.

For the second example, I got an error.

Could you help with checking if it's a bug?