search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
58 stars 29 forks source link

[Feature Request]: Calculate posterior probability of finding the effect whithin or outside the ROPE #1885

Open Masharipov opened 1 year ago

Masharipov commented 1 year ago

Description

In the JASP, Bayesian inference is perfomed using Bayes Factors. An alternative way of Bayesian inference can be to consider the posterior probability distribution (aka Bayesian paramerer inference). Is would be great to get tables with posterior probabilites of finding the effect (a) inside the region of practical equivalence (ROPE), (b) to the left of the ROPE and (c) to the right of the ROPE. Additionally, it would be convinient to get the log posterior odds (LPOs) for effects inside the ROPE, to the left of the ROPE and to the right of the ROPE. I.e. LPO(effect inside the ROPE) = ln(PostProb(effect inside the ROPE)/PostProb(effect outside the ROPE)). LogBF = Log posterior odds + Log prior odds

This procedure refers to the "ROPE-only" decision rule described in Kruschke (2018, see supplementary).

Purpose

To perform Bayesian parameter inference with the "ROPE-only" decision rule.

Use-case

All Bayesian t-tests

Is your feature request related to a problem?

No response

Describe the solution you would like

Add table with posterior probabilities of finding the effect inside and outside (to the left and to the right) the ROPE. And table with posterior probabilities converted to the natural logarithm of posterior odds (LPOs).

Describe alternatives that you have considered

No response

Additional context

Kruschke, J. K. (2018). Rejecting or Accepting Parameter Values in Bayesian Estimation. Advances in Methods and Practices in Psychological Science, 1(2), 270–280. https://doi.org/10.1177/2515245918771304

P.S. Sorry for creating multiple features requests (it was reccomended to create a single request per issue in JASP forum).

juliuspfadt commented 1 year ago

@Masharipov, thanks for the request. It is indeed better for the programmers to have single issues with single requests. Same as before, I will assign the maintainer!

Masharipov commented 1 year ago

@juliuspf, thank you! Sorry for creating two requests with similar titles. Can you please also see this one: https://github.com/jasp-stats/jasp-issues/issues/1882

EJWagenmakers commented 1 year ago

Is this what we provide in the module "Equivalence t-tests"?

Masharipov commented 1 year ago

@EJWagenmakers Yes, the module "Equivalence t-tests" is very useful and gives almost all the necessary information.

As far as I understand, this module reports PostProb inside the ROPE and outside the ROPE. It would be great to additionaly add PostProb to the left and to the right of the ROPE (to look for positive and negative effects separately). It would be also convinient to additionaly convert PostProb to Log Posterior Odds (LPOs) (to see difference between effects with very high PostProb, e.g. 0.991 and 0.999). LPOnull = ln(PostProb(effect inside the ROPE)/PostProb(effect outside the ROPE)). LPOpos = ln(PostProb(to the right of the ROPE)/PostProb(inside the ROPE + to the left of the ROPE)). LPOneg = ln(PostProb(to the left of the ROPE)/PostProb(inside the ROPE + to the right of the ROPE)).

In the current release, the ROPE can only be specified in Cohen's d. It would be convenient to also specify ROPE in raw units (please, see https://github.com/jasp-stats/jasp-issues/issues/1883)

And also, it would be great to see Credible Intervals for a given prior in Descriptives (in the "Equivalence t-tests" module) (see https://github.com/jasp-stats/jasp-issues/issues/1882).