search

mml-book / mml-book.github.io

Companion webpage to the book "Mathematics For Machine Learning"
13.24k stars 2.44k forks source link

Orthogonal feature matrix #456

Closed CL-BZH closed 4 years ago

CL-BZH commented 4 years ago

Is your feature request related to a problem? Please describe. In order to use formula (9.71) one need an orthonormal basis not just an orthogonal basis. So, the last 2 sentences of page 314 are a bit misleading.

Describe the solution you'd like After the first sentence of page 315 ("When the basis is not orthogonal, one can convert a set of linearly independent basis functions to an orthogonal basis by using the Gram-Schmidt process (Strang, 2003)." add something like "Then by normalizing the basis, one obtain an "orthonormal" (orthogonal) feature matrix."

mpd37 commented 4 years ago

(9.71) states that "if the basis is orthonormal" then that equation simplifies.

The sentences below are along the lines that we can choose any basis and turn it into an orthogonal basis, so that we can interpret this as an orthogonal projection. Orthonormality is not required here.

CL-BZH commented 4 years ago

Hi Marc, Thanks for your reply. The sentences are still in the paragraph started with "If the basis is orthonormal..." and the way it is written one may think that all what is needed in order to use formula (9.71) is to transform a basis into an orthogonal basis. Also, "the columns of Φ form an orthonormal basis ... so that the coupling between different features has disappeared" could be reworded because the decoupling is due only to the orthogonality (normality is not needed). Well, you may think I'm too picky so feel free to close this issue... :) Chris.

mpd37 commented 4 years ago

Good points. What do you think about this:

image

CL-BZH commented 4 years ago

Perfect. Thank you. (should I close the issue or you do it?)

mpd37 commented 4 years ago

I'll close it later. Thanks!