topepo / caret

caret (Classification And Regression Training) R package that contains misc functions for training and plotting classification and regression models
http://topepo.github.io/caret/index.html
1.61k stars 634 forks source link

Cost not affecting my SVM model accuracy - Caret #1339

Closed susiemg closed 11 months ago

susiemg commented 1 year ago

I am trying to build an SVM model that predicts whether a number is even or odd with a polynomial kernel. I have varied my cost values to see if there would be a drastic effect but there is no difference in accuracy. Surely there is something wrong?

This is my code:

> ctrl <- trainControl(method = "cv", number = 5)
> poly_grid <- expand.grid(degree = c(2, 3, 5), C = 10^seq(-6, 6, length.out=5), scale=1)
> 
> set.seed(1)
> poly_model <- train(evenodd ~ ., data = train, method = "svmPoly", 
> trControl = ctrl, tuneGrid = poly_grid)

These are my results

Support Vector Machines with Polynomial Kernel 

303 samples
784 predictors
2 classes: 'even', 'odd' 

Pre-processing: scaled (784) 
Resampling: Cross-Validated (4 fold) 
Summary of sample sizes: 228, 228, 227, 226 
Resampling results across tuning parameters:

degree  C      Accuracy   Kappa    
2       1e-06  0.7919982  0.5820659
2       1e-03  0.8645477  0.7290415
2       1e+00  0.8645477  0.7290415
2       1e+03  0.8645477  0.7290415
2       1e+06  0.8645477  0.7290415
3       1e-06  0.8511728  0.7020653
3       1e-03  0.8511728  0.7020653
3       1e+00  0.8511728  0.7020653
3       1e+03  0.8511728  0.7020653
3       1e+06  0.8511728  0.7020653
5       1e-06  0.7587964  0.5154335
5       1e-03  0.7587964  0.5154335
5       1e+00  0.7587964  0.5154335
5       1e+03  0.7587964  0.5154335
5       1e+06  0.7587964  0.5154335

Tuning parameter 'scale' was held constant at a value of 1
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were degree = 2, scale = 1 and C = 0.001.