search

DanburyAI / SG_DLB_2017

Repo for our deep learning book study group launched in October 2017.
9 stars 6 forks source link

prince book exercises #14

Open rogovski opened 7 years ago

rogovski commented 7 years ago

this commit needs check

Andrewnetwork commented 6 years ago

Looking through the diff is a bit hard. Maybe linking to file is a better procedure? I'm not sure, just a point.

Andrewnetwork commented 6 years ago

This is a verification issue. The ID is 14. You can use the hash symbol to pull up issue ID's. See #15 for discussion on the verification process.

14

Andrewnetwork commented 6 years ago

This notebook: https://github.com/DanburyAI/SG_DLB_2017/blob/master/Notebooks/MemberNBS/ProbabilityAndStats/Exercise%20-%20Prince's%20CV%20book%202.1%20to%202.5.ipynb

Andrewnetwork commented 6 years ago

2,1 Is solid. P(X) is Bernoulli, P(y) is multi-variate Gaussian. The question is asking "where x is a discrete random variable and where y is a continuous random variable."

Andrewnetwork commented 6 years ago

2.3 There's an issue displaying the latex in github due to using the \begin{align} ... \end{align} tags. The $$ .. $$ symbols in ipython denote this tag, so if you explicitly use them, they become redundant and do not render in some renderers -- like github.

Andrewnetwork commented 6 years ago

2.2 Is incorrect I believe. In order to marginalize out variables w,y you would need to sum over in the discrete case, or integrate over in the continuous case, these two variables. In the case of two variables where we marginalize one See https://en.wikipedia.org/wiki/Marginal_distribution capture

rogovski commented 6 years ago

@Andrewnetwork regarding the comment about the latex rendering issue. See the same notebook from my fork; I have removed the \begin{align} ... \end{align} - it still does not render. Have you gotten equation rendering to work in any of your notebooks just using $$..$$?

Andrewnetwork commented 6 years ago

@rogovski I'll have to experiment. I see that the tags I mentioned aren't the only issue. Also line numbering doesn't work in the github viewer. Perhaps we can just link to the notebook viewer in the future and abandon githubs renderings of the notebooks because it doesn't do it well

https://nbviewer.jupyter.org/github/rogovski/SG_DLB_2017/blob/master/Notebooks/MemberNBS/ProbabilityAndStats/Exercise%20-%20Prince%27s%20CV%20book%202.1%20to%202.5.ipynb

Andrewnetwork commented 6 years ago

2.5

Question: If variables x and y are independent and variables x and z are independent, does it follow that variables y and z are independent?

Proof by counter example.

Suppose it does follow that Y and Z are independent from the given independence of X,Y and X,Z. Then whenever this condition holds, Y and Z must be independent; however we can derive a case where the condition holds and Y and Z are not independent.

X and Y are in independent. X and Z are independent. Does it follow Y and Z are independent? No.

QED

Sometimes Y,Z are independent, but it does not follow from X,Y and X,Z being independent -- as shown above. If it is given that y and z are independent, P(Y,Z) = P(Y)*P(Z), you can derive this from the given information by:

Proof by definition.

We have: P(X,Y) = P(X)P(Y) (1) P(X,Z) = P(X)P(Z) (2) P(Y,Z) = P(Y) * P(Z) (3)

Show that P(Y,Z) = P(Y)*P(Z) (3) follows from statements (1) and (2).

By dividing both sides of (1) by P(X): P(X,Y)\P(X) = P(Y) (4) By dividing both sides of (2) by P(Z): P(X,Z)\P(X) = P(Z) (5)

P(Y,Z) = P(Y) P(Z) = P(X,Y)\P(X) P(X,Z)\P(Z)

QED