Open bob-carpenter opened 10 years ago
Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.
With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.
We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.
Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?
On 12/10/13, 11:43 AM, bgoodri wrote:
Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.
With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.
— Reply to this email directly or view it on GitHub https://github.com/stan-dev/stan/issues/426#issuecomment-30244232.
I'm not sure if we have anything useful to say in the manual that Stan supports at this point. Basically, if someone has a structured matrix as a parameter of their model, then they have to construct it the brute force way for the most part. I think it is not even possible to assign to a block of a matrix in the Stan language.
In any event, my main point was just to brainstorm whether batch updates would be an alternative to a single, giant, structured mass matrix in the momentum part of the model.
On Tue, Dec 10, 2013 at 3:31 PM, Bob Carpenter notifications@github.comwrote:
We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.
Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?
- Bob
On 12/10/13, 11:43 AM, bgoodri wrote:
Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.
With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.
— Reply to this email directly or view it on GitHub < https://github.com/stan-dev/stan/issues/426#issuecomment-30244232>.
— Reply to this email directly or view it on GitHubhttps://github.com/stan-dev/stan/issues/426#issuecomment-30264913 .
It'd be nice to have structured mass matrices so that we could have a mass matrix where there are structured zeros. The use case is when we think there are correlations in the posterior among some of the parameters that should go into the mass matrix. The particular use case motivating this is a model with a dozen or so parameters and then a whole lot of item-level parameters (what some people might call "random effects" if they wanted to annoy Andrew).