How to modify SVR loss function?

cjlin1 / libsvm

LIBSVM -- A Library for Support Vector Machines

https://www.csie.ntu.edu.tw/~cjlin/libsvm/

BSD 3-Clause "New" or "Revised" License

4.52k stars 1.64k forks source link

How to modify SVR loss function? #206

Closed iN8mare closed 9 months ago

iN8mare commented 9 months ago

I am working on a research project and I want to change the usual loss function of SVR by simply adding a hyperparameter that can penalize certain predicted values more compared to others.

For example, if the predicted value is greater than some k, then i want to increase the loss function by some hyperparameter alpha.

How can i implement or change this? I tried looking up in scikit learn but it's not possible to do this. Any help would be appreciated.

cjlin1 commented 9 months ago

It may not be easy to do this in libsvm. You must derive the dual problem first and see if the current solver can solve the new optimization problem. You might want to write the loss as a continuous and almost differentiable function, and then apply a tool with autodiff (e.g., pytorch)