stan-dev / rstanarm

rstanarm R package for Bayesian applied regression modeling
https://mc-stan.org/rstanarm
GNU General Public License v3.0
388 stars 132 forks source link

Feature: more flexibility in specifying global scale in the HS-prior #114

Closed jpiironen closed 7 years ago

jpiironen commented 8 years ago

Summary:

I would like a possibility of adjusting the scale and degrees of freedom for the global parameter tau in the hierarchical shrinkage (HS) prior.

Description:

We discussed the use of HS-prior for regression models in our technical report (Piironen and Vehtari, 2015) after which it was added to rstanarm, which is great! However, the current version allows only tuning the degrees of freedom for the local scale parameters lambda_j, but does not allow changing the prior for the global parameter tau which is given half-Cauchy prior with scale 1. For some datasets, this prior for tau gives too much mass for implausibly large values, and therefore favors solutions with many of the regression coefficients different from zero (I'm working on a paper that would give a recommendation on how to specify prior for tau based on our guess for the number of relevant variables in the data). A suggestion:

1) Replace tau ~ half-Cauchy(0,1) with tau ~ half-t_nu(0,s), and allow user to change the degrees of freedom nu and scale s (by default they can be set to 1 and 1, however).

2) In regression, there are theoretical considerations why one might want to use prior tau ~ half-t_nu(0,s*sigma), where sigma is the noise std. So it would be nice to have this alternative too.

I believe item 1) should be rather easy and quick to implement but would be very useful by providing more control over tau. However, item 2) might require more extra work and there might be some practical difficulties in the implementation and use I'm not aware of. Thus I'm proposing it only as something you could consider in the future, and I think item 1) would be more important to have at the moment.

Thanks in advance!

References:

J. Piironen, A. Vehtari (2015). Projection predictive variable selection using Stan+R, http://arxiv.org/abs/1508.02502

jgabry commented 8 years ago

Hi Juho, thanks for the suggestion. It seems like a good idea to me. @bgoodri what do you think?

For (1) we could preserve backwards compatibility if we add a second argument to hs, something like hs(df=3, global_scale = 1).

For (2) I agree that some more research is probably needed, but the coding shouldn't be too bad.

On Monday, September 5, 2016, Juho Piironen notifications@github.com wrote:

Summary:

I would like a possibility of adjusting the scale and degrees of freedom for the global parameter tau in the hierarchical shrinkage (HS) prior. Description:

We discussed the use of HS-prior for regression models in our technical report (Piironen and Vehtari, 2015) after which it was added to rstanarm, which is great! However, the current version allows only tuning the degrees of freedom for the local scale parameters lambda_j, but does not allow changing the prior for the global parameter tau which is given half-Cauchy prior with scale 1. For some datasets, this prior for tau gives too much mass for implausibly large values, and therefore favors solutions with many of the regression coefficients different from zero (I'm working on a paper that would give a recommendation on how to specify prior for tau based on our guess for the number of relevant variables in the data). A suggestion:

1) Replace tau ~ half-Cauchy(0,1) with tau ~ half-t_nu(0,s), and allow user to change the degrees of freedom nu and scale s (by default they can be set to 1 and 1, however).

2) In regression, there are theoretical considerations why one might want to use prior tau ~ half-t_nu(0,s*sigma), where sigma is the noise std. So it would be nice to have this alternative too.

I believe item 1) should be rather easy and quick to implement but would be very useful by providing more control over tau. However, item 2) might require more extra work and there might be some practical difficulties in the implementation and use I'm not aware of. Thus I'm proposing it only as something you could consider in the future, and I think item 1) would be more important to have at the moment.

Thanks a lot! References:

J. Piironen, A. Vehtari (2015). Projection predictive variable selection using Stan+R, http://arxiv.org/abs/1508.02502

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114, or mute the thread https://github.com/notifications/unsubscribe-auth/AHb4Q8Vep4ZTkLV1TP31kDyHqT3Ak0EWks5qm8vPgaJpZM4J008J .

bgoodri commented 8 years ago

Yeah

On Mon, Sep 5, 2016 at 4:17 PM, Jonah Gabry notifications@github.com wrote:

Hi Juho, thanks for the suggestion. It seems like a good idea to me. @bgoodri what do you think?

For (1) we could preserve backwards compatibility if we add a second argument to hs, something like hs(df=3, global_scale = 1).

For (2) I agree that some more research is probably needed, but the coding shouldn't be too bad.

On Monday, September 5, 2016, Juho Piironen notifications@github.com wrote:

Summary:

I would like a possibility of adjusting the scale and degrees of freedom for the global parameter tau in the hierarchical shrinkage (HS) prior. Description:

We discussed the use of HS-prior for regression models in our technical report (Piironen and Vehtari, 2015) after which it was added to rstanarm, which is great! However, the current version allows only tuning the degrees of freedom for the local scale parameters lambda_j, but does not allow changing the prior for the global parameter tau which is given half-Cauchy prior with scale 1. For some datasets, this prior for tau gives too much mass for implausibly large values, and therefore favors solutions with many of the regression coefficients different from zero (I'm working on a paper that would give a recommendation on how to specify prior for tau based on our guess for the number of relevant variables in the data). A suggestion:

1) Replace tau ~ half-Cauchy(0,1) with tau ~ half-t_nu(0,s), and allow user to change the degrees of freedom nu and scale s (by default they can be set to 1 and 1, however).

2) In regression, there are theoretical considerations why one might want to use prior tau ~ half-t_nu(0,s*sigma), where sigma is the noise std. So it would be nice to have this alternative too.

I believe item 1) should be rather easy and quick to implement but would be very useful by providing more control over tau. However, item 2) might require more extra work and there might be some practical difficulties in the implementation and use I'm not aware of. Thus I'm proposing it only as something you could consider in the future, and I think item 1) would be more important to have at the moment.

Thanks a lot! References:

J. Piironen, A. Vehtari (2015). Projection predictive variable selection using Stan+R, http://arxiv.org/abs/1508.02502

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114, or mute the thread https://github.com/notifications/unsubscribe-auth/ AHb4Q8Vep4ZTkLV1TP31kDyHqT3Ak0EWks5qm8vPgaJpZM4J008J .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-244807051, or mute the thread https://github.com/notifications/unsubscribe-auth/ADOrqjNl4mcSP9ByZzBITibUv34vwjmnks5qnHjlgaJpZM4J008J .

jgabry commented 8 years ago

@bgoodri looks like you reparameterized this in

https://github.com/stan-dev/rstanarm/blob/master/inst/chunks/priors_glm.stan

at some point. I'm sure I could figure it out but do you know off the top of your head what I'd need to tweak there to set there that would equate to changing the half-Cauchy scale parameter?

On Monday, September 5, 2016, bgoodri notifications@github.com wrote:

Yeah

On Mon, Sep 5, 2016 at 4:17 PM, Jonah Gabry <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Hi Juho, thanks for the suggestion. It seems like a good idea to me. @bgoodri what do you think?

For (1) we could preserve backwards compatibility if we add a second argument to hs, something like hs(df=3, global_scale = 1).

For (2) I agree that some more research is probably needed, but the coding shouldn't be too bad.

On Monday, September 5, 2016, Juho Piironen <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Summary:

I would like a possibility of adjusting the scale and degrees of freedom for the global parameter tau in the hierarchical shrinkage (HS) prior. Description:

We discussed the use of HS-prior for regression models in our technical report (Piironen and Vehtari, 2015) after which it was added to rstanarm, which is great! However, the current version allows only tuning the degrees of freedom for the local scale parameters lambda_j, but does not allow changing the prior for the global parameter tau which is given half-Cauchy prior with scale 1. For some datasets, this prior for tau gives too much mass for implausibly large values, and therefore favors solutions with many of the regression coefficients different from zero (I'm working on a paper that would give a recommendation on how to specify prior for tau based on our guess for the number of relevant variables in the data). A suggestion:

1) Replace tau ~ half-Cauchy(0,1) with tau ~ half-t_nu(0,s), and allow user to change the degrees of freedom nu and scale s (by default they can be set to 1 and 1, however).

2) In regression, there are theoretical considerations why one might want to use prior tau ~ half-t_nu(0,s*sigma), where sigma is the noise std. So it would be nice to have this alternative too.

I believe item 1) should be rather easy and quick to implement but would be very useful by providing more control over tau. However, item 2) might require more extra work and there might be some practical difficulties in the implementation and use I'm not aware of. Thus I'm proposing it only as something you could consider in the future, and I think item 1) would be more important to have at the moment.

Thanks a lot! References:

J. Piironen, A. Vehtari (2015). Projection predictive variable selection using Stan+R, http://arxiv.org/abs/1508.02502

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114, or mute the thread https://github.com/notifications/unsubscribe-auth/ AHb4Q8Vep4ZTkLV1TP31kDyHqT3Ak0EWks5qm8vPgaJpZM4J008J .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/stan-dev/rstanarm/issues/114#issuecomment-244807051 , or mute the thread https://github.com/notifications/unsubscribe-auth/ ADOrqjNl4mcSP9ByZzBITibUv34vwjmnks5qnHjlgaJpZM4J008J .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-244807905, or mute the thread https://github.com/notifications/unsubscribe-auth/AHb4Q68_7LiUC6Xys24ts_D25axlFnthks5qnHsVgaJpZM4J008J .

bgoodri commented 8 years ago

It is Matt tricked, so look at hs_prior in

https://github.com/stan-dev/rstanarm/blob/master/inst/chunks/common_functions.stan

On Mon, Sep 5, 2016 at 4:29 PM, Jonah Gabry notifications@github.com wrote:

@bgoodri looks like you reparameterized this in

https://github.com/stan-dev/rstanarm/blob/master/inst/ chunks/priors_glm.stan

at some point. I'm sure I could figure it out but do you know off the top of your head what I'd need to tweak there to set there that would equate to changing the half-Cauchy scale parameter?

On Monday, September 5, 2016, bgoodri notifications@github.com wrote:

Yeah

On Mon, Sep 5, 2016 at 4:17 PM, Jonah Gabry <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Hi Juho, thanks for the suggestion. It seems like a good idea to me. @bgoodri what do you think?

For (1) we could preserve backwards compatibility if we add a second argument to hs, something like hs(df=3, global_scale = 1).

For (2) I agree that some more research is probably needed, but the coding shouldn't be too bad.

On Monday, September 5, 2016, Juho Piironen <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');>

wrote:

Summary:

I would like a possibility of adjusting the scale and degrees of freedom for the global parameter tau in the hierarchical shrinkage (HS) prior. Description:

We discussed the use of HS-prior for regression models in our technical report (Piironen and Vehtari, 2015) after which it was added to rstanarm, which is great! However, the current version allows only tuning the degrees of freedom for the local scale parameters lambda_j, but does not allow changing the prior for the global parameter tau which is given half-Cauchy prior with scale 1. For some datasets, this prior for tau gives too much mass for implausibly large values, and therefore favors solutions with many of the regression coefficients different from zero (I'm working on a paper that would give a recommendation on how to specify prior for tau based on our guess for the number of relevant variables in the data). A suggestion:

1) Replace tau ~ half-Cauchy(0,1) with tau ~ half-t_nu(0,s), and allow user to change the degrees of freedom nu and scale s (by default they can be set to 1 and 1, however).

2) In regression, there are theoretical considerations why one might want to use prior tau ~ half-t_nu(0,s*sigma), where sigma is the noise std. So it would be nice to have this alternative too.

I believe item 1) should be rather easy and quick to implement but would be very useful by providing more control over tau. However, item 2) might require more extra work and there might be some practical difficulties in the implementation and use I'm not aware of. Thus I'm proposing it only as something you could consider in the future, and I think item 1) would be more important to have at the moment.

Thanks a lot! References:

J. Piironen, A. Vehtari (2015). Projection predictive variable selection using Stan+R, http://arxiv.org/abs/1508.02502

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114, or mute the thread https://github.com/notifications/unsubscribe-auth/ AHb4Q8Vep4ZTkLV1TP31kDyHqT3Ak0EWks5qm8vPgaJpZM4J008J .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/stan-dev/rstanarm/issues/114# issuecomment-244807051 , or mute the thread https://github.com/notifications/unsubscribe-auth/ ADOrqjNl4mcSP9ByZzBITibUv34vwjmnks5qnHjlgaJpZM4J008J .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/stan-dev/rstanarm/issues/114#issuecomment-244807905 , or mute the thread https://github.com/notifications/unsubscribe- auth/AHb4Q68_7LiUC6Xys24ts_D25axlFnthks5qnHsVgaJpZM4J008J

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-244808176, or mute the thread https://github.com/notifications/unsubscribe-auth/ADOrqhv-vyNo8v2Ng2zttzZUQld3h_JUks5qnHvAgaJpZM4J008J .

jgabry commented 8 years ago

Ok, so to only allow the scale of the Cauchy prior on tau to be set then it should as simple as adding another argument global_prior_scale to hs_prior() in that file and then changing the last line of hs_prior from

return z_beta .* lambda * global[1] * sqrt(global[2]);

to

return z_beta .* lambda * global[1] * sqrt(global[2]) * global_prior_scale;

correct?

If so, then I think then also allowing df to be changed (i.e. moving from cauchy(0, global_prior_scale) to student_t(global_prior_df, 0, global_prior_scale)) would then only require also changing in https://github.com/stan-dev/rstanarm/blob/master/inst/chunks/priors_glm.stan

target += inv_gamma_lpdf(global[2] | 0.5, 0.5);

to

target += inv_gamma_lpdf(global[2] | 0.5 * global_prior_df, 0.5 * global_prior_df);

@bgoodri does that seem right to you?

bgoodri commented 8 years ago

@avehtari reminded me of this. What @jgabry wrote makes sense. Did you have a branch / commit that did so?

jgabry commented 8 years ago

Yeah I think I have a local branch. I'm walking at the moment but will check later and make the branch public

On Friday, October 14, 2016, bgoodri notifications@github.com wrote:

@avehtari https://github.com/avehtari reminded me of this. What @jgabry https://github.com/jgabry wrote makes sense. Did you have a branch / commit that did so?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-253840436, or mute the thread https://github.com/notifications/unsubscribe-auth/AHb4Q5EOQwzcOVRIrsYeVsNtg-9C34HDks5qz6TXgaJpZM4J008J .

jgabry commented 8 years ago

@bgoodri just pushed the branch I was working on a while ago. take a look at https://github.com/stan-dev/rstanarm/tree/feature-more-flexible-hs-prior

ariddell commented 8 years ago

Would be nice to have the prior mentioned in https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations

jpiironen commented 7 years ago

Hi! I just wanted to ask about the current situation of this feature. I noticed there is a feature branch for this but I don't know if its stable/ready yet. Thanks.

jgabry commented 7 years ago

I think @bgoodri said he would finish this at some point. I'm not sure of the current status.

On Tue, Nov 29, 2016 at 3:18 AM Juho Piironen notifications@github.com wrote:

Hi! I just wanted to ask about the current situation of this feature. I noticed there is a feature branch for this but I don't know if its stable/ready yet. Thanks.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-263505021, or mute the thread https://github.com/notifications/unsubscribe-auth/AHb4QyVpYuhSwovfFkHAF1Bj_fcXQklPks5rC9_qgaJpZM4J008J .

bgoodri commented 7 years ago

In the hsplus prior, should there also be a multiplication by a new global_scale parameter and by sigma in the Gaussian case?

jgabry commented 7 years ago

I'll defer to Finland on this one

jpiironen commented 7 years ago

Yes, I think the prior for tau should be the same in both hs and hsplus, that is, the user can change the scale and degrees of freedom, and in the Gaussian case the scale is proportional to sigma.

jgabry commented 7 years ago

@bgoodri Did all of this get implemented in the last release? If so let's close this. If not, what else still needs to be done?

bgoodri commented 7 years ago

Yeah

On Wed, Feb 1, 2017 at 3:39 PM, Jonah Gabry notifications@github.com wrote:

@bgoodri https://github.com/bgoodri Did all of this get implemented in the last release? If so let's close this. If not, what else still needs to be done?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stan-dev/rstanarm/issues/114#issuecomment-276775199, or mute the thread https://github.com/notifications/unsubscribe-auth/ADOrqpSepsTDWERJmEkKQHzSSUDkPwzyks5rYO2PgaJpZM4J008J .