Problem SVM in the final layer of CNN

sdemyanov / ConvNet

Convolutional Neural Networks for Matlab for classification and segmentation, including Invariang Backpropagation (IBP) and Adversarial Training (AT) algorithms. Trained on GPU, require cuDNN v5.

240 stars 141 forks source link

Problem SVM in the final layer of CNN #8

Closed mnaetr closed 9 years ago

mnaetr commented 10 years ago

Hi. I'm using ConvNet with medical images. Using the code without SVM in the final layer I had good results. Then I tried to introduce the SVM and the results are very strange. With values of the C constant equal to 1, 10, 100, 1000 I always obtain a training error of 1.4e+4 and a test error of 50%. What can I do? In the cnnbp.m function i found a commented line in which C is present. I have to uncomment it?

sdemyanov commented 10 years ago

Hi. The problem with SVM is that the derivatives are not constrained to be in [-1,1]. This is why you should use the learning rate 10-100 times less than usual. Actually, I haven't observe the situation when it had been better than regular sigmoid, but you can try.