search

alashworth / test-issue-import

0 stars 0 forks source link

structured mass matrices #7

Open alashworth opened 5 years ago

alashworth commented 5 years ago

Issue by bob-carpenter Saturday Nov 30, 2013 at 17:00 GMT Originally opened as https://github.com/stan-dev/stan/issues/426

It'd be nice to have structured mass matrices so that we could have a mass matrix where there are structured zeros. The use case is when we think there are correlations in the posterior among some of the parameters that should go into the mass matrix. The particular use case motivating this is a model with a dozen or so parameters and then a whole lot of item-level parameters (what some people might call "random effects" if they wanted to annoy Andrew).

alashworth commented 5 years ago

Comment by bgoodri Tuesday Dec 10, 2013 at 16:43 GMT

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

alashworth commented 5 years ago

Comment by bob-carpenter Tuesday Dec 10, 2013 at 20:31 GMT

We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.

Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

— Reply to this email directly or view it on GitHub https://github.com/stan-dev/stan/issues/426#issuecomment-30244232.

alashworth commented 5 years ago

Comment by bgoodri Tuesday Dec 10, 2013 at 22:20 GMT

I'm not sure if we have anything useful to say in the manual that Stan supports at this point. Basically, if someone has a structured matrix as a parameter of their model, then they have to construct it the brute force way for the most part. I think it is not even possible to assign to a block of a matrix in the Stan language.

In any event, my main point was just to brainstorm whether batch updates would be an alternative to a single, giant, structured mass matrix in the momentum part of the model.

On Tue, Dec 10, 2013 at 3:31 PM, Bob Carpenter notifications@github.comwrote:

We should probably mention this in the manual w.r.t. user modeling, where the user can easily break things apart if the matrix block structure is known, for instance.

Banded structure's even trickier --- my understanding is that it's not uncommon in something like Gaussian processes due to hard thresholding on distances of effects. Is there a way to decompose these in our modeling language?

  • Bob

On 12/10/13, 11:43 AM, bgoodri wrote:

Is it easier / better to go with structured mass matrices or batched updates of parameters that have a batch-specific mass matrix? In your example, you could alternate between updating the dozen parameters with a dense mass matrix and the item-level parameters with a diagonal mass matrix. I guess this would require us to implement parameter tagging.

With a single mass matrix, it seems like you would need to use an Eigen::SparseMatrix or some sort of block matrix scheme. But there currently is no eigenvalues algorithm for an Eigen::SparseMatrix, so we couldn't integrate this with RMHMC, unless you did fill-in, which would be expensive.

— Reply to this email directly or view it on GitHub < https://github.com/stan-dev/stan/issues/426#issuecomment-30244232>.

— Reply to this email directly or view it on GitHubhttps://github.com/stan-dev/stan/issues/426#issuecomment-30264913 .