When t is very extreme, one sided Bayes factors don't add up

[richarddmorey]

set.seed(1)
xx = rnorm(100) 
bf10 = as.vector(ttestBF(xx))
bfp0 = as.vector(ttestBF(xx, nullInterval = c(0,Inf)))[1]
bfm0 = as.vector(ttestBF(xx, nullInterval = c(0,Inf)))[2]

bf10
> Alt., r=0.707 
> 0.2251773 
(bfp0 + bfm0)/2
> Alt., r=0.707 0<d<Inf 
> 0.2251773

When t is not extreme, the point null Bayes factor is equal to the average of the one-sided Bayes factors. However, when t is extreme:

xx = xx + .4
bf10 = as.vector(ttestBF(xx))
bfp0 = as.vector(ttestBF(xx, nullInterval = c(0,Inf)))[1]
bfm0 = as.vector(ttestBF(xx, nullInterval = c(0,Inf)))[2]

bf10
> Alt., r=0.707 
> 86630.19 
(bfp0 + bfm0)/2
> Alt., r=0.707 0<d<Inf 
> 125485.8 

...this is not the case. The integration and approximation must be breaking down for large t.

[richarddmorey]

This was fixed in latest release (0.9.11).

[Gijsp]

Thank you for your work on this wonderful package.

I'm using release 0.9.12-2 but am still getting what looks like a similar error.

dat1<-c(0.007653674, 0.008777279, 0.004344134, 0.011565081, 0.011367040, 0.011746161) dat2<-c(0.03366234, 0.02315733, 0.01014003, 0.02873154, 0.02555487, 0.02832875) ttestBF(dat2, dat1, paired=T) # all is fine ttestBF(dat2, dat1, paired=T, nullInterval=c(0, Inf)) # t is large; approximation invoked And the returned values don't make sense.

I hope this msg helps making the package even better.

[richarddmorey]

Hi Gijsp, can you describe why the result doesn't make sense? I get:

> BFInfo()
Package BayesFactor
0.9.12-2
> dat1<-c(0.007653674, 0.008777279, 0.004344134, 0.011565081, 0.011367040, 0.011746161)
> dat2<-c(0.03366234, 0.02315733, 0.01014003, 0.02873154, 0.02555487, 0.02832875)
> ttestBF(dat2, dat1, paired=T)
Bayes factor analysis
--------------
[1] Alt., r=0.707 : 23.70152 ±0%

Against denominator:
  Null, mu = 0
---
Bayes factor type: BFoneSample, JZS

> bf = ttestBF(dat2, dat1, paired=T, nullInterval=c(0, Inf))
> bf
Bayes factor analysis
--------------
[1] Alt., r=0.707 0<d<Inf    : 47.35625   ±NA%
[2] Alt., r=0.707 !(0<d<Inf) : 0.04679239 ±NA%

Against denominator:
  Null, mu = 0
---
Bayes factor type: BFoneSample, JZS
t is large; approximation invoked.
t is large; approximation invoked.

> mean(as.vector(bf))
[1] 23.70152

...which seems to make sense. Can you paste your output here?

[Gijsp]

Thank you for checking, I realize the problem only occurs for some values in nullInterval, smaller than Inf.

For my data I know that dat2-dat1 can never exceed 1, so I did:

ttestBF(dat2, dat1, paired=T, nullInterval=c(0, 1)) t is large; approximation invoked. t is large; approximation invoked.

Bayes factor analysis

[1] Alt., r=0.707 0<d<1 : 0.6253687 ±NA% [2] Alt., r=0.707 !(0<d<1) : 33.78489 ±NA%

But from re-reading the documentation these reverse results are probably due to not specifying the null interval in 'standardized units'. I guess it's not possible to define the nullinterval in the same units as the data. Is that correct?

[richarddmorey]

Yes, that's correct; the prior is placed on the standardized effect size. Due to the non-informative prior on the variance, the corresponding prior on the unstandardized difference is not proper. This has the benefit of making the test more generally applicable, but the drawback of not being able to define the prior (or Bayes factor) on the units of the original variable.

[Gijsp]

Thanks for the clear explanation!