search

stan-dev / stan

Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
2.59k stars 369 forks source link

structured mass matrices #426

Open bob-carpenter opened 10 years ago

bob-carpenter commented 10 years ago

It'd be nice to have structured mass matrices so that we could have a mass matrix where there are structured zeros. The use case is when we think there are correlations in the posterior among some of the parameters that should go into the mass matrix. The particular use case motivating this is a model with a dozen or so parameters and then a whole lot of item-level parameters (what some people might call "random effects" if they wanted to annoy Andrew).

bgoodri commented 10 years ago

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

bob-carpenter commented 10 years ago

We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.

Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

— Reply to this email directly or view it on GitHub https://github.com/stan-dev/stan/issues/426#issuecomment-30244232.

bgoodri commented 10 years ago

I'm not sure if we have anything useful to say in the manual that Stan supports at this point. Basically, if someone has a structured matrix as a parameter of their model, then they have to construct it the brute force way for the most part. I think it is not even possible to assign to a block of a matrix in the Stan language.

In any event, my main point was just to brainstorm whether batch updates would be an alternative to a single, giant, structured mass matrix in the momentum part of the model.

On Tue, Dec 10, 2013 at 3:31 PM, Bob Carpenter notifications@github.comwrote:

We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.

Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?

  • Bob

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

— Reply to this email directly or view it on GitHub < https://github.com/stan-dev/stan/issues/426#issuecomment-30244232>.

— Reply to this email directly or view it on GitHubhttps://github.com/stan-dev/stan/issues/426#issuecomment-30264913 .