Feature request: Descriptive CIs from model

AugustOlsson commented 4 years ago

Enhancement: Include the possibilty to get descriptive plots with Credible intervals that are not based on a uniform prior (e.g. include the possibility to get CI from the model one has fitted)
Purpose: To get credible intervals which reflect the data and that use the power of the bayesian framework (credible intervals with uniform prior over the entire line is pretty much equal to a confidence interval I think?)
Use-case: Usefull in all scenarions. But to exemplify, when using a dependent variable that by definition cannot take a negative value (e.g. RMSE), it is very strange to get credible intervals that include negative values (due to participants being very good in some cells). I think this would not be the case if the CI was based on the model.

Is your feature request related to a problem? Please describe. Due to the current set-up of how CIs are caluclated one can get credible intervals that include impossible values. This makes the current CIs less usefull and more difficult to interpret.

Describe the solution you'd like E.g. include the option to get CIs based on the model.

PS. I asked about this on the forums way back, but then forgot to make it an enhancement request: https://forum.cogsci.nl/discussion/5761/impossible-values-included-in-credible-interval#latest

boutinb commented 4 years ago

@EJWagenmakers To whom could we assign this feature request?

AlexanderLyNL commented 4 years ago

The CI for descriptives are based on the uniform prior, because they are to be used to describe the data only. In the analyses you can find posterior plots, there the shown credible intervals are based on other priors.

AugustOlsson commented 4 years ago

Hi Alexander @AlexanderLyNL ,

When perfoming a bayesian ANOVA I can indeed get model averaged posteriors distribution and/or single model posterior distribution, with density on the y-axis. But there is (as far as I can find) no option to get a plot of the data (i.e. differences in the dependent variable, with the scale of the dependent variable on the y-axis) where the CI are based on information from the model.

tomtomme commented 9 months ago

@AugustOlsson Is this still an issue?

AugustOlsson commented 8 months ago

@tomtomme I'll download the new version during next week and check

/August

AugustOlsson commented 8 months ago

@AugustOlsson Is this still an issue?

Hi, I just installed jasp 0.18.3 and there seems to be a major bug, because now I can event make graphs with 2 or more variables. Rather I can only put a variable in the "horisontal axis" but I can't add the other variable to the "separate lines" box. Hence I cannot check if CI:s are now based on the fitted model.

I actually need to make some graphs of this sort right now. Is it possible to downgrade to a previous version so that I can make the graphs?

jasp bug

tomtomme commented 8 months ago

Sure, you can always uninstall and install older versions from here: https://jasp-stats.org/previous-versions

tomtomme commented 8 months ago

@AugustOlsson I checked this with anova and have not problem putting in another factor into separate lines field. Just select it and use the lower arow button. If this really does not work, can you share your jasp file, by renaming to .zip and drag and drop here?

Here my jasp file that shows, that it works for me at least: example.jasp.zip

AugustOlsson commented 8 months ago

@tomtomme

I found out what the problem is. If I remake the independent variables so that their content is numerical (i.e they are still marked as nominal but have numeric content) than I can plot the interaction. However, if the nominal variables are string variables (i.e. have the verbal names of the conditions) then one cannot plot the interaction. Would you like the data to check this for yourself or do you have enough to go on?

Best, August

AugustOlsson commented 8 months ago

Regarding the original issue, as far as I can see it still works the same. That is, credible intervals are based are always based on the uniform prior and there is no option to get credible intervals based on the model predictions.

Should I create a new issue for the bug?

AugustOlsson commented 8 months ago

I posted a bug-report on the graphic issue here: https://github.com/jasp-stats/jasp-issues/issues/2573

tomtomme commented 8 months ago

@AugustOlsson Thx for the update. This is still vailid. I can however give no estimate on when the option to choose another prior for CIs might be implemented.

relevant reference from duplicate https://github.com/jasp-stats/jasp-issues/issues/1882

Kruschke, J. K. (2018). Rejecting or Accepting Parameter Values in Bayesian Estimation. Advances in Methods and Practices in Psychological Science, 1(2), 270–280. https://doi.org/10.1177/2515245918771304 Note that https://github.com/jasp-stats/jasp-issues/issues/1882 wants this implemented non-uniform-prior-CIs as a table, while this issues wants them plottet

jasp-stats / jasp-issues

Feature request: Descriptive CIs from model #785