> The idea here is that we want to isolate x from the relation y = Ax. Under some mild assumptions we can do this by y = Ax => A^Ty = A^T A x => (A^T A)^{-1} A^T y = x

mml-book / mml-book.github.io

Companion webpage to the book "Mathematics For Machine Learning"

13.11k stars 2.42k forks source link

> The idea here is that we want to isolate x from the relation y = Ax. Under some mild assumptions we can do this by y = Ax => A^Ty = A^T A x => (A^T A)^{-1} A^T y = x #769

Open arya-map22 opened 8 months ago

arya-map22 commented 8 months ago

          > The idea here is that we want to isolate x from the relation y = Ax. Under some mild assumptions we can do this by y = Ax => A^Ty = A^T A x => (A^T A)^{-1} A^T y = x

I'm not sure whether this helps?

Yes, but in the 6.91 it said that the E(X) = (A^T A)^{-1} A^T y, with y a state of random variable Y. And from definition of Y, E(Y)=Ax (page 204) not y = Ax (page 203), so i got confused here. How come the mean of X become like that?

Originally posted by @arya-map22 in https://github.com/mml-book/mml-book.github.io/issues/398#issuecomment-1891008047

arya-map22 commented 8 months ago

Could you give me another reference for that specific method?

mpd37 commented 8 months ago

It may make more sense if we wrote p(y|x) in (6.89) and p(x|y) in (6.91) ?

arya-map22 commented 8 months ago

Using the formula from 6.66 and 6.67 with the definition of y = Ax, E(x) = mu, Cov(x,x) = sigma, E(y) = Amu, and Cov(y,y) = Asigma*A^T (just like showed on the page 203). I get E(y|x) = Ax, but with covariance matrix (y|x) = 0. Doing the same for the (x|y), i get E(y|x) = (A)^{-1}y (then replacing the inverse with moore-penrose pseudo inverse we get the same as stated in the book), but again i get cov(x|y) = 0. Where did i go wrong?