ageron / handson-ml3

A series of Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in Python using Scikit-Learn, Keras and TensorFlow 2.
Apache License 2.0
7.84k stars 3.13k forks source link

[BUG] chapter 5, Setting the parameter C to float("inf") doesn't work. #73

Open ArezooAalipanah opened 1 year ago

ArezooAalipanah commented 1 year ago

Thanks for helping us improve this project!

Before you create this issue Please make sure you are using the latest updated code and libraries: see https://github.com/ageron/handson-ml3/blob/main/INSTALL.md#update-this-project-and-its-libraries

Also please make sure to read the FAQ (https://github.com/ageron/handson-ml3#faq) and search for existing issues (both open and closed), as your question may already have been answered: https://github.com/ageron/handson-ml3/issues

Describe the bug On chapter 5 for the code of "Figure 5-1. Large margin classification" there is one part on google colab where the C parameter is set to float("inf"), however, when I run the program I receive an error indicating that its not acceptable to use inf page 176- google colab

To Reproduce Please copy the code that fails here, using code blocks like this:

# extra code – this cell generates and saves Figure 5–1

import matplotlib.pyplot as plt
import numpy as np
from sklearn.svm import SVC
from sklearn import datasets

iris = datasets.load_iris(as_frame=True)
X = iris.data[["petal length (cm)", "petal width (cm)"]].values
y = iris.target

setosa_or_versicolor = (y == 0) | (y == 1)
X = X[setosa_or_versicolor]
y = y[setosa_or_versicolor]

# SVM Classifier model
svm_clf = SVC(kernel="linear", C=float("inf")) ##this is the part with the issue
svm_clf.fit(X, y)

# Bad models
x0 = np.linspace(0, 5.5, 200)
pred_1 = 5 * x0 - 20
pred_2 = x0 - 1.8
pred_3 = 0.1 * x0 + 0.5

def plot_svc_decision_boundary(svm_clf, xmin, xmax):
    w = svm_clf.coef_[0]
    b = svm_clf.intercept_[0]

    # At the decision boundary, w0*x0 + w1*x1 + b = 0
    # => x1 = -w0/w1 * x0 - b/w1
    x0 = np.linspace(xmin, xmax, 200)
    decision_boundary = -w[0] / w[1] * x0 - b / w[1]

    margin = 1/w[1]
    gutter_up = decision_boundary + margin
    gutter_down = decision_boundary - margin
    svs = svm_clf.support_vectors_

    plt.plot(x0, decision_boundary, "k-", linewidth=2, zorder=-2)
    plt.plot(x0, gutter_up, "k--", linewidth=2, zorder=-2)
    plt.plot(x0, gutter_down, "k--", linewidth=2, zorder=-2)
    plt.scatter(svs[:, 0], svs[:, 1], s=180, facecolors='#AAA',
                zorder=-1)

fig, axes = plt.subplots(ncols=2, figsize=(10, 2.7), sharey=True)

plt.sca(axes[0])
plt.plot(x0, pred_1, "g--", linewidth=2)
plt.plot(x0, pred_2, "m-", linewidth=2)
plt.plot(x0, pred_3, "r-", linewidth=2)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs", label="Iris versicolor")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo", label="Iris setosa")
plt.xlabel("Petal length")
plt.ylabel("Petal width")
plt.legend(loc="upper left")
plt.axis([0, 5.5, 0, 2])
plt.gca().set_aspect("equal")
plt.grid()

plt.sca(axes[1])
plot_svc_decision_boundary(svm_clf, 0, 5.5)
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs")
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo")
plt.xlabel("Petal length")
plt.axis([0, 5.5, 0, 2])
plt.gca().set_aspect("equal")
plt.grid()

save_fig("large_margin_classification_plot")
plt.show()

And if you got an exception, please copy the full stacktrace here:

InvalidParameterError                     Traceback (most recent call last)
[<ipython-input-11-ad9449b51ae6>](https://localhost:8080/#) in <cell line: 18>()
     16 # SVM Classifier model
     17 svm_clf = SVC(kernel="linear", C=float("inf"))
---> 18 svm_clf.fit(X, y)
     19 
     20 # Bad models

2 frames
[/usr/local/lib/python3.9/dist-packages/sklearn/utils/_param_validation.py](https://localhost:8080/#) in validate_parameter_constraints(parameter_constraints, params, caller_name)
     95                 )
     96 
---> 97             raise InvalidParameterError(
     98                 f"The {param_name!r} parameter of {caller_name} must be"
     99                 f" {constraints_str}. Got {param_val!r} instead."

InvalidParameterError: The 'C' parameter of SVC must be a float in the range (0.0, inf). Got inf instead.

Screenshots image

image

Versions (please complete the following information): all codes and versions set in this code: https://colab.research.google.com/github/ageron/handson-ml3/blob/main/05_support_vector_machines.ipynb#scrollTo=bkZHEPZwizro

masalome1025 commented 1 year ago

I have the same issue, but unable to find out why. I have tried to debug and it seemed the bug is from "_param_validation.py" it is failed at when do the "right_cmp". But the watch window showed that the value of "val" and "right" should be same which is "inf"

print screen 2023-05-29 003811