Question on statistical tests for `betabin()`

[KilianSander]

Bi (2006, Section 7.4) describes two statistical tests for the corrected beta-binomial model:

1. Null and alternative hypotheses for a replicated difference test are $H_0:\mu=0,\gamma=0$ and $H_1:\mu>0,\gamma>0$. The test statistic is calculated from $\hat{\mu}$, $\hat{\gamma}$, and their covariance matrix from the ML estimation and asymptotically follows a $\chi^2$ distribution with two degrees of freedom.
2. Difference testing for a specified probability of correct responses with the null and alternative hypotheses $H_0:\pi_c=\pi_0$ and $H_1:\pi_c>\pi_0$. The test statistic is $Z=\frac{\hat{\pi_c}-\pi_0}{\sqrt{V\left(\hat{\pi_c}\right)_0}}$, where $V\left(\hat{\pi_c}\right)_0$ is the variance under the null hypothesis. $Z$ asymptotically follows a standard normal distribution

How do these tests relate to the LR tests of overdispersion and association as implemented in summary.betabin()?

References Bi, J. (2006). Sensory discrimination tests and measurements: Statistical principles, procedures and tables (1st ed.). Blackwell.

[dominik-aigora]

Hi Kilian, thanks for reaching out.

I am maintaining this package on behalf of the Aigora team. We took over the maintenance of the package from the original authors to ensure its availability and functionality.

However, given the complexity of the statistical models and the codebase we believe that without direct insight from the original authors it is difficult to find out the exact relationship between these tests and the implementation in package. We are intending to look into it ourselves but we would strongly recommend to contact the authors directly.

[KilianSander]

Hi Dominik,

thanks for your reply.

I will contact the authors and get back to you with their response.

[runehaubo]

Dear Kilian,

The short answer is that they are unrelated - or at least that they are derived from a different perspective (I don't want to rule out that asymptotic equivalences can be derived...)

The help page/manual entry describes the implemented likelihood ratio tests as follows:

'The summary method provides a likelihood ratio test of over-dispersion on one degree of freedom and a likelihood ratio test of association (i.e. where the null hypothesis is "no difference" and the alternative hypothesis is "any difference") on two degrees of freedom (chi-square tests).'

Thus the latter test is a test of both mean difference and dispersion.

Hope this helps Rune

On Thu, 22 Feb 2024 at 13:07, Kilian Sander @.***> wrote:

Bi (2006, Section 7.4) describes two statistical tests for the corrected beta-binomial model:

1. Null and alternative hypotheses for a replicated difference test are $H_0:\mu=0,\gamma=0$ and $H_1:\mu>0,\gamma>0$. The test statistic is calculated from $\hat{\mu}$, $\hat{\gamma}$, and their covariance matrix from the ML estimation and asymptotically follows a $\chi^2$ distribution with two degrees of freedom.
2. Difference testing for a specified probability of correct responses with the null and alternative hypotheses $H_0:\pi_c=\pi_0$ and $H_1:\pi_c>\pi_0$. The test statistic is $Z=\frac{\hat{\pi_c}-\pi_0}{\sqrt{V\left(\hat{\pi_c}\right)_0}}$, where $V\left(\hat{\pi_c}\right)_0$ is the variance under the null hypothesis. $Z$ asymptotically follows a standard normal distribution

How do these tests relate to the LR tests of overdispersion and association as implemented in summary.betabin()?

References Bi, J. (2006). Sensory discrimination tests and measurements: Statistical principles, procedures and tables (1st ed.). Blackwell.

— Reply to this email directly, view it on GitHub https://github.com/aigorahub/sensR/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZZLOARMH3WYJADI2TCX3YU4YGVAVCNFSM6AAAAABDU2EZ3GVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2DQOJSGE3TQMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[KilianSander]

Hi Rune,

thanks for your reply.

If I understand correctly, the LR test of association and the first test described by Bi are dealing with the same things (mean difference, $\mu$, and dispersion, $\gamma$). Thus, they may answer the question of whether the data are significantly different from the guessing model.

I also looked at the code in R/betaBin.R. Am I right that the LR test of association "compares" the log-likelihoods of the fitted model (res$logLik) and the binomial guessing model (res$logLikNull, which constraints $\mu$ and $\gamma$ to 0), while the LR test of overdispersion does this for the fitted model and a binomial model, where the probability of success is the overall proportion of correct responses (res$logLikMu which only constraints $\gamma$ to 0, but estimates $\mu$ from the data)? https://github.com/aigorahub/sensR/blob/865116f632a349d4508f3b66d8e2826a691e6370/R/betaBin.R#L49-L55 https://github.com/aigorahub/sensR/blob/865116f632a349d4508f3b66d8e2826a691e6370/R/betaBin.R#L174-L179

Kilian

[runehaubo]

Yes, you are absolutely right!

Cheers Rune

tirs. 27. feb. 2024 kl. 15.46 skrev Kilian Sander @.***

:

Hi Rune,

thanks for your reply.

If I understand correctly, the LR test of association and the first test described by Bi are dealing with the same things (mean difference, $\mu$, and dispersion, $\gamma$). Thus, they may answer the question of whether the data are significantly different from the guessing model.

I also looked at the code in R/betaBin.R. Am I right that the LR test of association "compares" the log-likelihoods of the fitted model (res$logLik) and the binomial guessing model (res$logLikNull, which constraints $\mu$ and $\gamma$ to 0), while the LR test of overdispersion does this for the fitted model and a binomial model, where the probability of success is the overall proportion of correct responses (res$logLikMu which only constraints $\gamma$ to 0, but estimates $\mu$ from the data)?

https://github.com/aigorahub/sensR/blob/865116f632a349d4508f3b66d8e2826a691e6370/R/betaBin.R#L49-L55

Kilian

— Reply to this email directly, view it on GitHub https://github.com/aigorahub/sensR/issues/4#issuecomment-1966713461, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHZZLMS4JDENOK75B4MNETYVXWT5AVCNFSM6AAAAABDU2EZ3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWG4YTGNBWGE . You are receiving this because you commented.Message ID: @.***>