Matrix A is useless

[hahamyt]

After I comment out the following code in KernelKSVD.m

% for J=1:maxIter
%     disp(J);
%     A=Anew;
%     % SPARSE CODING STAGE
%     for q =1:size(Y,2)
%         [X(:,q)] = KOMP(Y(:,q),B,A,T0,kfnc,KBB);
%     end
%     
%     X(isnan(X))=0;
%     X(isinf(X))=0;
%     
%     % A UPDATE STAGE
%     for k = 1:K
%         drop_k=1:K;
%         wk=find(X(k,:));
%         Ek=(eye(size(Y,2))-A(:,drop_k)*X(drop_k,:));
%         EkR=Ek(:,wk);
%         [U,S,~]=svd(EkR'*KBB*EkR,'econ');
%         if isempty(S)
%            Anew(:,k)=0;
%         else
%            Anew(:,k)=1/sqrt(S(1,1))*EkR*U(:,1);
%         end
%         
%     end
%     
%     [r] = evalKKSVD(Y,Y,Anew,T0,kfnc,KBB,X);
%     if mean(abs(r))<=errThresh
%         break
%     end

% end

The test result is better than before.

But why?

The performance is good enough with a random A and no need for further learning...

[hilikliming]

I tried to reproduce what you described this morning however I still get better performance when letting A get updated.

Which dataset were you working with when you observed this?

On MNIST I get about 95% correct classification when updating A as is written. When commenting out the A updates like you suggested I got about 94% CC (still very high, not what I would've expected!). On XOR I got 88% CC with random A and 92% with updated A.

I wonder if this good performance with a random coeff matrix A is related to Random Projections/these Extreme Learning Approaches... hmm. I'll continue to look into this when I have time, I haven't touched this paper in a while but this is an interesting observation for sure!

[hahamyt]

Thank you for your excellent code and reply, I test the code on mnist.mat.