Closed kiudee closed 9 years ago
I wonder if that's related to the Wishart distribution for which some issues have been reported.
The same error happens when I input a fixed precision matrix. As soon as I get home, I'll see if I can fix it.
As far as I understand it, the problem is that logp(self, value)
expects a single observation as input (then the output is the correct log probability).
Instead in model.py
it gets called with the complete array:
self.logp_elemwiset = distribution.logp(*args)
Since I’m not yet familiar with the inner workings of PyMC, what is the expected input and output of the logp
function?
This would be my first draft for the logp
function:
def logp(self, value):
mu = self.mu
tau = self.tau
delta = value - mu
k = tau.shape[0]
n = value.shape[0]
result = n * k * log(2*pi) + n * log(det(tau))
result += (delta.T * dot(tau, delta.T)).sum()
return -1/2. * result
Which works for this version of the model:
testdata = np.random.multivariate_normal([1,2,3], np.eye(3),size=1000)
with pm.Model() as model:
mu = pm.Normal('mu', mu=0, tau=5 ** -2, shape=n_col)
like = pm.MvNormal('likelihood', mu=mu, tau=pm.constant(np.eye(3)), observed=testdata)
The traces for mu
can be seen here:
Sadly, this stops working as soon as I include the Wishart prior. I’ll look at it more closely.
Found another bug: tau
is a precision matrix, therefore we need to take the inverse of the determinant:
def logp(self, value):
mu = self.mu
tau = self.tau
delta = value - mu
k = tau.shape[0]
n = value.shape[0]
result = n * k * log(2*pi) + n * log(1./det(tau))
result += (delta.dot(tau) * delta).sum()
return -1/2. * result
After delving a bit deeper into the code of PyMC 3, I realized that the log probabilities are always computed element wise.
The adjusted version of MvNormal.logp
then looks like:
def logp(self, value):
mu = self.mu
tau = self.tau
delta = value - mu
k = tau.shape[0]
result = k * log(2*pi) + log(1./det(tau))
result += (delta.dot(tau) * delta).sum(axis=1)
return -1/2. * result
Sadly, even with my fixes to the Wishart distribution the sampler won’t converge to anything remotely useful. I’m open to suggestions, as I’ve already looked over the functions several times. (Note: The problems only occur, when I include the Wishart prior distribution.)
Update: It has to be the logp of the Wishart distribution. I implemented the complete model in emcee and the multivariate normal distribution works fine. As soon as I add the log probability of the Wishart distribution, it returns nonsensical results.
Looks like great progress @kiudee. I think @kadeng did some fixes to Wishart in his PR (that's not merge as it has many other parts): https://github.com/kadeng/pymc/blob/master/pymc/distributions/multivariate.py
Perhaps that fixes the remaining issues?
Thanks for the hint! I‘ll look at it. By the way, I did just implement the Lewandowski, Kurowicka and Joe (LKJ) prior distribution for correlation matrices (Quick overview on CrossValidated). So far it works great in emcee, and I plan to implement it for PyMC soon.
I matched my code to @kadeng's but I still have the same problem. His code also still calls the multivariate Gamma function with swapped parameters.
The problem (which also occurs in emcee with the Wishart) is that the log probability and the values in the matrix just keep exploding.
Related: https://github.com/pymc-devs/pymc/issues/538 https://github.com/pymc-devs/pymc/issues/395
In emcee you implemented your own Wishart?
What about the one in pymc2?
I just looked at the fortran code of the Wishart distribution (pymc2). I noticed it checks if the matrix is symmetric and positive definite.
Yes, I ported the code to emcee to rule out any sampler causes.
But you're saying the Wishart doesn't work in emcee either, right?
Perhaps the half-normal T prior is also worth looking at: http://ba.stat.cmu.edu/journal/2013/vol08/issue02/huang.pdf
Can you put your current changes up on a branch somewhere?
Yes, I did not get the Wishart prior to work. The LKJ Prior for the correlation matrix works already. As soon as I have a clean version in PyMC I'll create a separate pull request.
I created a pull request for the fix of the original problem (‘Matrices not aligned’) in #675.
I have been converting a model of mine from PyMC 2 to version 3 and I’m encountering a problem in the calculation of the log probability of MvNormal.
In my model
data
is an n_row x n_col matrix (each row containing an observation of the multivariate normal):When running this example, an error occurs while calculating the dot product
dot(delta.T, dot(tau, delta))
:I’m on the following version of Theano:
Here is the complete error: