search

eriklindernoren / ML-From-Scratch

Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.
MIT License
23.56k stars 4.55k forks source link

Suppport Vector Machine Problem #41

Open tchaton opened 6 years ago

tchaton commented 6 years ago

I had implemented my own svm based on your implementation. Wierdly, the accuracy was very low. I copied your code and use sklearn has a benchemark. Your svm implementation gives me back almost random prediction.

Here is the code for you to check.

https://github.com/tchaton/interviews_prep/blob/master/code_prep/mlfromscratch/supervised/svm.py

tchaton commented 6 years ago

And idx = lagr_mult > 1e-7 create an error too using the last dataset. I modified it to idx = lagr_mult > 1e-11. But it isn t a good solution.

akhilvasvani commented 5 years ago

1) Did you normalize your data before you used rbf kernel?

2) In your example, there are 3 classes (0, 1, 2). If you removed one class (say 0) and then ran the implementation, your accuracy would improve greatly.

2) On this website (https://pythonprogramming.net/soft-margin-kernel-cvxopt-svm-machine-learning-tutorial/), the code is pretty identical with a couple of slight differences.. anyhow, running this code with with how you created your dataset yielded a WORSE accuracy. On the bottom of the page, the author wrote that this SVM code is meant to display the inner workings and not for robust usage. So similarly, this is applied here.