Use Y instead of b in matrix multiplication example

[johnwmillr]

Thanks for this list, it's very helpful!

This is a nitpicky suggestion which you can take or leave. I suggest you use Y instead of b in your ridge regression example. The difference term, ||AX-b||, is comparing the output of the model, AX, to the true output value, which I think is more commonly labeled Y rather than b.

Here's the Wikipedia page on ridge regression for reference.

My change:

Matrix multiplication with @

Let's implement one of the simplest ML models — a linear regression with l2 regularization (a.k.a. ridge regression):

# l2-regularized linear regression: || AX - Y ||^2 + alpha * ||x||^2 -> min

# Python 2
X = np.linalg.inv(np.dot(A.T, A) + alpha * np.eye(A.shape[1])).dot(A.T.dot(Y))
# Python 3
X = np.linalg.inv(A.T @ A + alpha * np.eye(A.shape[1])) @ (A.T @ Y)

[983]

It is preferable to use np.linalg.lstsq instead of calculating an inverse matrix.

[arogozhnikov]

Sorry for this delay. I've just had a look into books, both 'Elements of Statistical Learning' and 'Machine Learning: A Probabilistic Perspective' use non-capital y (which sounds more reasonable to me).