deepgenerativemodels / notes

Course notes
MIT License
554 stars 121 forks source link

The number of parameters needed to specify a table of a Bayesian network #33

Open JadonChan opened 3 months ago

JadonChan commented 3 months ago

This is from Autoregressive Models Chapter

To see why, let us consider the conditional for the last dimension, given by $p(xn|x{\lt n})$. In order to fully specify this conditional, we need to specify a probability for $2^{n−1}$ configurations of the variables $x_1,x2,\ldots,x{n−1}$. Since the probabilities should sum to $1$, the total number of parameters for specifying this conditional is given by $2^{n−1}−1$. Hence, a tabular representation for the conditionals is impractical for learning the joint distribution factorized via chain rule.

Shouldn't it be $2^{n-1}$ instead of $2^{n-1}-1$ here ? Why minus $1$ ? In my understanding, the $n$-th random variable is dependent on $n-1$ random variables, in binary case, there should be $2^{n-1}$ rows in the table. In every single row, the entries should add up to $1$, so only one of the two entries in this row needs specifying. Thus one parameter for each row, it should be $2^{n-1}$.

chrisoffner commented 1 month ago

I came across the same issue. It seems that the LaTeX has already been corrected but the website does not reflect this change yet: https://github.com/deepgenerativemodels/notes/blob/bd2303339eaaea884870125b473cc1ae8c980d51/autoregressive/index.tex#L16C426-L16C518