search

alashworth / test-issue-import

0 stars 0 forks source link

naturally parameterized multivariate normal #26

Open alashworth opened 5 years ago

alashworth commented 5 years ago

Issue by bob-carpenter Friday Jan 03, 2014 at 23:33 GMT Originally opened as https://github.com/stan-dev/stan/issues/482

From Andrew:

Add a paraemterization of the multivariate normal that takes the parameters Sigma^{-1}*mu and Sigma^{-1} (i.e., the so-called natural parameters of the exponential family) rather than mu and Sigma?

I'm asking because, in this MC-EP thing I'm playing around with, I find myself doing a lot of inverting matrices and solving linear systems in order to get mu and Sigma to pass in to my prior distribution. It's no big deal in terms of "wall time" (the dimensionality is low), I was just wondering.

alashworth commented 5 years ago

Comment by jgabry Friday Mar 04, 2016 at 18:32 GMT

Is this still something that's wanted or can we close this?

alashworth commented 5 years ago

Comment by avehtari Friday Mar 04, 2016 at 20:28 GMT

It would be helpful for "EP as a a way of life"

alashworth commented 5 years ago

Comment by syclik Wednesday Nov 30, 2016 at 14:52 GMT

@avehtari and @andrewgelman: please come up with a name for the function and the function signature. Once we have that, we can create an issue in math to actually create the function and get this going.

alashworth commented 5 years ago

Comment by avehtari Wednesday Nov 30, 2016 at 15:24 GMT

normal_natural or normal_nat

same naming could be used in theory for other exponential family distributions in the future

alashworth commented 5 years ago

Comment by syclik Wednesday Nov 30, 2016 at 15:29 GMT

That's not enough info. Can you actually provide the function signature too? I'm looking for something like: real foo(real bar, Matrix baz)

I'm vetoing _nat -- that isn't a good abbreviation since nat means something: https://en.wikipedia.org/wiki/Nat_(unit)

alashworth commented 5 years ago

Comment by avehtari Wednesday Nov 30, 2016 at 16:01 GMT

Following notation in EP-life paper real normal_natural(real r, Matrix Q)

alashworth commented 5 years ago

Comment by bob-carpenter Wednesday Nov 30, 2016 at 19:58 GMT

Following notation in EP-life paper

real normal_natural(real r, Matrix Q)

Only the Q parameter? So it's

r ~ normal_natural(Q);

Or should there be a location-like parameter?

The way we've been naming variant distributions is either by number or by their parameters, so multi_normal_prec for precision parameterization or exp_log for taking the mean on a log scale. Do the parameters have names?

alashworth commented 5 years ago

Comment by avehtari Wednesday Nov 30, 2016 at 20:27 GMT

Ooops (making mistakes while trying to answer too quickly)

real multi_normal_natural_lpdf(vectors y | vectors r, Matrix Q)

and same variations as for multi_normal_lpdf

r and Q are jointly called natural parameters. For normal distribution Q is same as precision, but r doesn't have a commonly used name. I just tried to find any name for it, but couldn't find any. All references I found refer to them jointly as natural parameters.

alashworth commented 5 years ago

Comment by syclik Wednesday Nov 30, 2016 at 20:37 GMT

Thanks. I was about to ask the same thing.

So... you want it to be vectorized for y and r, but not Q? So the actual function signatures would be: real multi_normal_natural_lpdf(vector y | vector r, Matrix Q) real multi_normal_natural_lpdf(vector[] y | vector r, Matrix Q) real multi_normal_natural_lpdf(vector y | vector[] r, Matrix Q) real multi_normal_natural_lpdf(vector[] y | vector[] r, Matrix Q)

Is that right?

On Wed, Nov 30, 2016 at 3:27 PM, Aki Vehtari notifications@github.com wrote:

Ooops (making mistakes while trying to answer, too, quickly)

real multi_normal_natural_lpdf(vectors y | vectors r, Matrix Q)

and same variations as for multi_normal_lpdf

r and Q are jointly called natural parameters. For normal distribution Q is same as precision, but r doesn't have a commonly used name. I just tried to find any name for it, but couldn't find any. All references I found refer to them jointly as natural parameters.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stan-dev/stan/issues/482#issuecomment-263985567, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZ_F0QVANdj1666Xw10z7L-BP2-5MXxks5rDdwtgaJpZM4BXXgD .

alashworth commented 5 years ago

Comment by avehtari Thursday Dec 01, 2016 at 08:21 GMT

I followed the Stan reference manual, which has real multi_normal_lpdf(vectors y | vectors mu, matrix Sigma) real multi_normal_lpdf(vectors y | row_vectors mu, matrix Sigma) real multi_normal_lpdf(row_vectors y | vectors mu, matrix Sigma) real multi_normal_lpdf(row_vectors y | row_vectors mu, matrix Sigma)

So the same function signature types for multi_normal_natural. I don't know why the reference has vectors, but you write vector[], and the reference has Matrix, but you write matrix.

Since real multi_normal_prec_lpdf(vectors y | vectors mu, matrix Omega) has Omega, it would be possible to name the arguments as real multi_normal_natural_lpdf(vectors y | vectors Omegamu, matrix Omega) where Omega is precision, but Omegamu doesn't have own name. (and the other functions signatures as for multi_normal and multi_normal_prec)

alashworth commented 5 years ago

Comment by syclik Thursday Dec 01, 2016 at 13:29 GMT

I was trying to be specific so I can write out an issue. The implementation is very different if it has to be vectorized or not. And the amount of work grows exponentially.

I don't know why multi_normal allows a mismatch between y and mu in terms of vectors and row_vectors, but we should avoid that if we can.

I wrote vector[] to mean an 1-d array of vectors.

On Dec 1, 2016, at 3:21 AM, Aki Vehtari notifications@github.com wrote:

I followed the Stan reference manual, which has real multi_normal_lpdf(vectors y | vectors mu, matrix Sigma) real multi_normal_lpdf(vectors y | row_vectors mu, matrix Sigma) real multi_normal_lpdf(row_vectors y | vectors mu, matrix Sigma) real multi_normal_lpdf(row_vectors y | row_vectors mu, matrix Sigma)

So the same function signature types for multi_normal_natural. I don't know why the reference has vectors, but you write vector[], and the reference has Matrix, but you write matrix.

Since real multi_normal_prec_lpdf(vectors y | vectors mu, matrix Omega) has Omega, it would be possible to name the arguments as real multi_normal_natural_lpdf(vectors y | vectors Omegamu, matrix Omega) where Omega is precision, but Omegamu doesn't have own name. (and the other functions signatures as for multi_normal and multi_normal_prec)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

alashworth commented 5 years ago

Comment by bob-carpenter Thursday Dec 01, 2016 at 22:24 GMT

On Dec 1, 2016, at 8:29 AM, Daniel Lee notifications@github.com wrote:

I was trying to be specific so I can write out an issue. The implementation is very different if it has to be vectorized or not. And the amount of work grows exponentially.

The coding work grows by some fixed difficulty level, but the time it takes to run the tests grows exponentially. Unless there's no testing framework, in which case it seems an impossible task to do it manually.

The coding just requires some wrapping in expression templates for vector views.

I don't know why multi_normal allows a mismatch between y and mu in terms of vectors and row_vectors, but we should avoid that if we can.

I'm not sure where that came from, but we'd been talking about allowing any kind of vectors with the right size. Users were running into problems with not being able to read our error messages and add transpositions.

I would also prefer to keep them coherent as just requiring a vector.

alashworth commented 5 years ago

Comment by syclik Thursday Dec 01, 2016 at 22:39 GMT

Yes, testing grows exponentially. I didn't write the vectorized version of the multi_normal, so I don't know if there were vector views that were easy to use in place. Now that I think about it, I would think there is. But, I'm thinking that for this to be useful, it needs to use every trick we know, which makes it harder to reason about than a single function signature that accepts (vector, vector, matrix).

On Thu, Dec 1, 2016 at 5:24 PM, Bob Carpenter notifications@github.com wrote:

On Dec 1, 2016, at 8:29 AM, Daniel Lee notifications@github.com wrote:

I was trying to be specific so I can write out an issue. The implementation is very different if it has to be vectorized or not. And the amount of work grows exponentially.

The coding work grows by some fixed difficulty level, but the time it takes to run the tests grows exponentially. Unless there's no testing framework, in which case it seems an impossible task to do it manually.

The coding just requires some wrapping in expression templates for vector views.

I don't know why multi_normal allows a mismatch between y and mu in terms of vectors and row_vectors, but we should avoid that if we can.

I'm not sure where that came from, but we'd been talking about allowing any kind of vectors with the right size. Users were running into problems with not being able to read our error messages and add transpositions.

I would also prefer to keep them coherent as just requiring a vector.

  • Bob

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stan-dev/stan/issues/482#issuecomment-264314457, or mute the thread https://github.com/notifications/unsubscribe-auth/AAZ_FzLvZ2HECo-0pugNKn0QUv73HMZuks5rD0kGgaJpZM4BXXgD .