Open syclik opened 9 years ago
From @rtrangucci on March 24, 2015 14:42
We still need first-order reverse-mode tests for matrix_normal_prec_log, unless I haven't found all the tests.
Right now there are three .cpp files that implement matrix_normal_prec_log tests:
As far as I can tell, none of these files implement reverse-mode first order tests. The fwd and mix files run first-order fvar<var>
and fvar<fvar<var>>
tests. We need a test/unit/math/rev/mat/prob/matrix_normal_test.cpp
that does the reverse-mode tests.
Implement one or more forms of the matrix normal distribution. The general distribution is of the form
where
SigmaRow
andSigmaCol
are positive definite covariance matrices. The covariance matrixSigmaRow
controls the covariance of the rows ofy
whereasSigmaRow
controls the covariances of the columns.There are three design decisions about how
SigmaRow
andSigmaCol
are parameterized, which are whether to useThere is also the issue of how to resolve the non-identifiability of scale between the two matrices, because
The log density can be expressed as follows, but this is not how we'd compute it.
If
LRow
andLCol
are Cholesky factors ofSigmaRow
andSigmaCol
so that, e.g.,SigmaRow = LRow * LRow'
, andz ~ normal(0, I)
, thenAs shown on the Wikipedia page for matrix normal, we can evaluate that trace term as
where
vec(y)
is the flattening ofy
. We then want to use the property of Kronecker products that@bgoodri suggested going further with Cholesky factorizations
LRow
andLCol
using the rules for inverse of the product of square matrixesinv(A * B) = inv(B) * inv(A)
and the trace of a product ruletr(A' * B) = vec(A)' * vec(B)
, to deriveIdentifiability will be left up to the user. One way to achieve it is to define one of the matrices to be a correlation matrix with a simplex of scales. That's most efficient when done with a simple product of a Cholesky factor rather than a quadratic form for the full correlation matrix. A second, more symmetric way to achieve a proper posterior is to decompose both covariance matrices into scales time Cholesky factors and put priors on both of the scales.
There is also a matrix Student-t distribution, which should just follow whatever matrix normal does.