search

mml-book / mml-book.github.io

Companion webpage to the book "Mathematics For Machine Learning"
13.21k stars 2.43k forks source link

More detailed description of final part of section 4.5.2 #583

Closed fbrundu closed 4 years ago

fbrundu commented 4 years ago

I don't know if others had the same problem, but probably a more detailed description of the last part of section 4.5.2 may be useful. For example, it is written (just before equation 4.77):

To reach this goal, we use the fact the images of the v_i under A have to be orthogonal, too.

I'm not sure what I'm missing here but aren't the right singular vectors computed by the SPD matrix A^T A, therefore they should be ONB for the SPD matrix. How can we use this result to show that their images under the A matrix are orthogonal?

Thanks

mpd37 commented 4 years ago

Assume matrix A has eigenvectors x_i, x_j (x_i and x_j are orthonormal). Then it holds that Ax_i is orthogonal to Ax_j because (Ax_i)^T (Ax_j) = \lambda_i\lambda_j x_i^T * x_j = 0 (because x_i, x_j are ONB).

fbrundu commented 4 years ago

It makes sense, thanks.