ICA is still getting poor recognition accuracy

[bentsherman]

I don't know if the bug has resurfaced or if it was never fixed, but ICA still tends to classify images under s10 with the ORL dataset. An example run:

$ ./face-rec --train train_images/ --test test_images/ --ica --ica_num_ic=50 --loglevel=2
Hyperparameters
PCA
  n1                            -1
LDA
  n1                            -1
  n2                            -1
ICA
  num_ic                        50
  max_iterations              1000
  epsilon                 0.000100
kNN
  k                              1

Training
  PCA
    compute surrogate of covariance matrix L
    compute eigenvectors of L
    compute PCA projection matrix
  ICA
    subtract mean from input matrix
    compute principal components of input matrix
    compute whitening matrix and whitened input matrix
    compute mixing matrix
    compute ICA projection matrix
Recognition
  ICA
    s10_2.pgm  -> s10  
    s10_4.pgm  -> s10  
    s10_5.pgm  -> s10  
    s11_2.pgm  -> s10  (!)
    s11_4.pgm  -> s10  (!)
    s11_5.pgm  -> s10  (!)
    s12_2.pgm  -> s12  
    s12_4.pgm  -> s10  (!)
    s12_5.pgm  -> s12  
    s13_2.pgm  -> s10  (!)
    s13_4.pgm  -> s10  (!)
    s13_5.pgm  -> s10  (!)
    s14_2.pgm  -> s10  (!)
    s14_4.pgm  -> s10  (!)
    s14_5.pgm  -> s10  (!)
    s15_2.pgm  -> s10  (!)
    s15_4.pgm  -> s10  (!)
    s15_5.pgm  -> s10  (!)
    s16_2.pgm  -> s10  (!)
    s16_4.pgm  -> s10  (!)
    s16_5.pgm  -> s10  (!)
    s17_2.pgm  -> s12  (!)
    s17_4.pgm  -> s10  (!)
    s17_5.pgm  -> s1   (!)
    s18_2.pgm  -> s10  (!)
    s18_4.pgm  -> s10  (!)
    s18_5.pgm  -> s10  (!)
    s19_2.pgm  -> s12  (!)
    s19_4.pgm  -> s10  (!)
    s19_5.pgm  -> s10  (!)
    s1_2.pgm   -> s10  (!)
    s1_4.pgm   -> s10  (!)
    s1_5.pgm   -> s10  (!)
    s20_2.pgm  -> s10  (!)
    s20_4.pgm  -> s10  (!)
    s20_5.pgm  -> s10  (!)
    s21_2.pgm  -> s10  (!)
    s21_4.pgm  -> s10  (!)
    s21_5.pgm  -> s10  (!)
    s22_2.pgm  -> s10  (!)
    s22_4.pgm  -> s10  (!)
    s22_5.pgm  -> s10  (!)
    s23_2.pgm  -> s10  (!)
    s23_4.pgm  -> s10  (!)
    s23_5.pgm  -> s10  (!)
    s24_2.pgm  -> s10  (!)
    s24_4.pgm  -> s10  (!)
    s24_5.pgm  -> s10  (!)
    s25_2.pgm  -> s10  (!)
    s25_4.pgm  -> s12  (!)
    s25_5.pgm  -> s12  (!)
    s26_2.pgm  -> s10  (!)
    s26_4.pgm  -> s12  (!)
    s26_5.pgm  -> s12  (!)
    s27_2.pgm  -> s12  (!)
    s27_4.pgm  -> s12  (!)
    s27_5.pgm  -> s12  (!)
    s28_2.pgm  -> s10  (!)
    s28_4.pgm  -> s10  (!)
    s28_5.pgm  -> s10  (!)
    s29_2.pgm  -> s10  (!)
    s29_4.pgm  -> s10  (!)
    s29_5.pgm  -> s10  (!)
    s2_2.pgm   -> s10  (!)
    s2_4.pgm   -> s10  (!)
    s2_5.pgm   -> s2   
    s30_2.pgm  -> s10  (!)
    s30_4.pgm  -> s10  (!)
    s30_5.pgm  -> s10  (!)
    s31_2.pgm  -> s10  (!)
    s31_4.pgm  -> s10  (!)
    s31_5.pgm  -> s10  (!)
    s32_2.pgm  -> s10  (!)
    s32_4.pgm  -> s10  (!)
    s32_5.pgm  -> s10  (!)
    s33_2.pgm  -> s10  (!)
    s33_4.pgm  -> s10  (!)
    s33_5.pgm  -> s10  (!)
    s34_2.pgm  -> s10  (!)
    s34_4.pgm  -> s10  (!)
    s34_5.pgm  -> s10  (!)
    s35_2.pgm  -> s12  (!)
    s35_4.pgm  -> s10  (!)
    s35_5.pgm  -> s10  (!)
    s36_2.pgm  -> s10  (!)
    s36_4.pgm  -> s10  (!)
    s36_5.pgm  -> s10  (!)
    s37_2.pgm  -> s10  (!)
    s37_4.pgm  -> s10  (!)
    s37_5.pgm  -> s10  (!)
    s38_2.pgm  -> s10  (!)
    s38_4.pgm  -> s10  (!)
    s38_5.pgm  -> s10  (!)
    s39_2.pgm  -> s10  (!)
    s39_4.pgm  -> s10  (!)
    s39_5.pgm  -> s10  (!)
    s3_2.pgm   -> s10  (!)
    s3_4.pgm   -> s12  (!)
    s3_5.pgm   -> s12  (!)
    s40_2.pgm  -> s10  (!)
    s40_4.pgm  -> s12  (!)
    s40_5.pgm  -> s10  (!)
    s4_2.pgm   -> s10  (!)
    s4_4.pgm   -> s10  (!)
    s4_5.pgm   -> s12  (!)
    s5_2.pgm   -> s10  (!)
    s5_4.pgm   -> s10  (!)
    s5_5.pgm   -> s10  (!)
    s6_2.pgm   -> s12  (!)
    s6_4.pgm   -> s10  (!)
    s6_5.pgm   -> s10  (!)
    s7_2.pgm   -> s10  (!)
    s7_4.pgm   -> s10  (!)
    s7_5.pgm   -> s12  (!)
    s8_2.pgm   -> s10  (!)
    s8_4.pgm   -> s12  (!)
    s8_5.pgm   -> s10  (!)
    s9_2.pgm   -> s10  (!)
    s9_4.pgm   -> s10  (!)
    s9_5.pgm   -> s10  (!)
    6 / 120 matched, 5.00%

[bentsherman]

ICA seems to be working properly during training, it produces similar matrices as the MATLAB code. I'm going to investigate db_recognize and the kNN classifier.

[bentsherman]

When I use the L2 distance for ICA I get similar recognition accuracy to the MATLAB code, which is good. However, I have found a bug with how kNN sorts neighbors by distance when using COS distance, so I will fix this first and then compare the distance functions for ICA.

[bentsherman]

The problem with the COS distance was actually with the kNN classifier; when comparing neighbors it would take the difference in distances and cast it to an integer, which means that differences between -1 and 1 would be rounded to 0. Since the COS distance returned values between -1 and 1, this case happened a lot, which caused the COS distance to yield poor results for all of the algorithms. I changed kNN to compare floating point values properly, and I also changed the COS distance to range from 0 to 2 so that it's more like a "distance".

At this point it seems that the COS distance might give comparable results to L2 for all three algorithms (at least for easy cases), so we'll definitely want to run some experiments that compare the distance functions across each algorithm.