search

aigamedev / scikit-neuralnetwork

Deep neural networks without the learning cliff! Classifiers and regressors compatible with scikit-learn.
BSD 3-Clause "New" or "Revised" License
1.21k stars 221 forks source link

Top 5 error is always zero? Matconvnet #225

Open AlsubaieNajah opened 7 years ago

AlsubaieNajah commented 7 years ago

Hi, I am using MatConvNet-beta18. I have 5 class each class has 34 sample (divided into training and testing group). The problem is that my top5 error is always zero? Does anyone have any idea why I am getting this and how to solve it. Thank you.

This is sample output: train: epoch 01: 1/ 2: 7.4 Hz obj:95.7 top1err:0.73 top5err:0 [100/100] train: epoch 01: 2/ 2: 9.0 Hz obj:87.4 top1err:0.706 top5err:0 [36/36] val: epoch 01: 1/ 1: 32.8 Hz obj:20.6 top1err:0.735 top5err:0 [34/34] cnn_train: saving model for epoch 1 cnn_train: model saved in 0.00021 s train: epoch 02: 1/ 2: 21.2 Hz obj:73.6 top1err:0.75 top5err:0 [100/100] train: epoch 02: 2/ 2: 23.2 Hz obj:83.8 top1err:0.75 top5err:0 [36/36] val: epoch 02: 1/ 1: 35.1 Hz obj:66.3 top1err:0.853 top5err:0 [34/34] cnn_train: saving model for epoch 2 cnn_train: model saved in 3e-05 s train: epoch 03: 1/ 2: 22.8 Hz obj:90.9 top1err:0.71 top5err:0 [100/100] train: epoch 03: 2/ 2: 24.5 Hz obj:93.4 top1err:0.684 top5err:0 [36/36] val: epoch 03: 1/ 1: 39.9 Hz obj:39.3 top1err:0.5 top5err:0 [34/34] cnn_train: saving model for epoch 3 cnn_train: model saved in 3.2e-05 s train: epoch 04: 1/ 2: 13.8 Hz obj:86.2 top1err:0.64 top5err:0 [100/100] train: epoch 04: 2/ 2: 14.4 Hz obj:89.6 top1err:0.632 top5err:0 [36/36] val: epoch 04: 1/ 1: 24.4 Hz obj:38.6 top1err:0.676 top5err:0 [34/34] cnn_train: saving model for epoch 4 cnn_train: model saved in 3.8e-05 s train: epoch 05: 1/ 2: 22.3 Hz obj:75.8 top1err:0.6 top5err:0 [100/100] train: epoch 05: 2/ 2: 24.0 Hz obj:80.2 top1err:0.588 top5err:0 [36/36] val: epoch 05: 1/ 1: 28.4 Hz obj:49.9 top1err:0.559 top5err:0 [34/34] cnn_train: saving model for epoch 5 cnn_train: model saved in 3e-05 s train: epoch 06: 1/ 2: 19.7 Hz obj:82.6 top1err:0.68 top5err:0 [100/100] train: epoch 06: 2/ 2: 21.2 Hz obj:72.5 top1err:0.632 top5err:0 [36/36] val: epoch 06: 1/ 1: 26.5 Hz obj:28.8 top1err:0.382 top5err:0 [34/34]

and it goes on similarly.. .

AlsubaieNajah commented 7 years ago

Hi, To explain the problem, In file cnn_train and inside the function error_multiclass. I noticed that I always get zero for this term (min(error(:,:,1:m,:),[],3))). The reason is because the computed error for each class is either 1 or zero and the minimum becomes zero.

error = ~bsxfun(@eq, predictions, labels) ; err(1,1) = sum(sum(sum(mass . error(:,:,1,:)))) ; err(2,1) = sum(sum(sum(mass . min(error(:,:,1:m,:),[],3)))) ;

Any help is appreciated.

Regards,

saskra commented 7 years ago

With five classes you will always have a top5err of zero, as you will always have a top10err of zero with ten classes etc.