vivamoto / classifier

Machine learning code, derivatives calculation and optimization algorithms developed during the Machine Learning course at Universidade de Sao Paulo. All codes in Python, NumPy and Matplotlib with example in the end of file.
15 stars 4 forks source link

There is a problem in function svm_predict() and the prediction cannot be implemented correctly #4

Open Cuisine4 opened 1 year ago

Cuisine4 commented 1 year ago

Describe the bug The problem appears on line 238 of file svm.py.In the original program, the formula for calculating the self.bias is as follows.

Compute the bias

    k = self.kernel(X_train, X_test)
    SV_neg = y_train < 0
    SV_pos = y_train > 0
    self.bias = (-1 / 2) * (np.max(k[SV_neg[:, 0], :].T @ alpha[SV_neg]) + np.min(k[SV_pos[:, 0], :].T @ alpha[SV_pos]))
    self.bias = y_train - np.sum(alpha * y_train * k, axis=1, keepdims=True)
    self.bias = np.mean(self.bias)

The bias calculated in this way is incorrect and will cause errors in later predictions

Expected behavior According to the formula I looked up, the correct calculation is as follows.

Compute the bias

    k = self.kernel(X_train, X_test)
    SV_neg = y_train < 0
    SV_pos = y_train > 0
    kk=self.kernel(X_train, X_train)
    self.bias = y_train - np.sum(alpha * y_train * kk, axis=1, keepdims=True)
    self.bias = np.mean(self.bias)

Screenshots Screenshot from the watermelon book "Machine learning" Zhou Zhihua section 6.2 image

github-actions[bot] commented 1 year ago

Thanks for submitting an issue.' first issue