search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
58 stars 29 forks source link

Bayesian Mann-Whitney unstable BF estimates #289

Closed AugustOlsson closed 5 years ago

AugustOlsson commented 5 years ago

I have two groups and a dependent variable. I am testing the hypothesis that the group differ.

When running the Bayesian Mann-Whitney U test the BF10 estimates wary a lot, from BF10 67 to BF10 450 over 10 runs. I am using 10 000 samples which seems to be the max. The W values is always the same, and so is the BF10 value from the Student test.

*Proposed fix If nothing is wrong with the actual calculations I propose increasing the max amount of sample to be able to find stable BF-factors.

Best, August ManWhitney test.zip

EJWagenmakers commented 5 years ago

@JohnnyDoorn ?

JohnnyDoorn commented 5 years ago

Hi August, The algorithm behind that analysis is indeed not as stable as the other Bayesian analyses, due to the data augmentation needed for the rank-based analyses. I was not aware that the limit on the number of samples is only 10,000 - I will increase this number and then hopefully it will be quite a bit more stable for you. However, there will always be somewhat of a variation and I'm currently working on an error % that gets included with the result (similar to the Bayesian ANOVA), in order to quantify the stability. Kind regards, Johnny

AugustOlsson commented 5 years ago

Hi Johanny,

Thanks for the quick reply. An error % which indicates stability would be very good. And ofc the possibility to run larger samples.

Best, August

AlexanderLyNL commented 5 years ago

Will be solved in the next release (0.9.3), see also issue #299