bvieth / powsimR

Power analysis is essential to optimize the design of RNA-seq experiments and to assess and compare the power to detect differentially expressed genes. PowsimR is a flexible tool to simulate and evaluate differential expression from bulk and especially single-cell RNA-seq data making it suitable for a priori and posterior power analyses.
https://bvieth.github.io/powsimR/
Artistic License 2.0
104 stars 23 forks source link

Which distributions are considered nested in `evaluateDist`? #45

Closed ekernf01 closed 4 years ago

ekernf01 commented 4 years ago

In evaluateDist, you use a likelihood ratio test when models are nested and a Vuong test otherwise. Which models do you consider to be nested, and for nested models, how do you determine the distribution of the likelihood ratio under the null?

From a newcomer's perspective, the statistics seem quite tricky. Reading some work by Paul Wilson (links below), it seems that the Poisson and ZIP are "nested" in the sense that the ZIP family includes the Poisson as a special case, but Wilks' Theorem does not apply because the special case happens when the zero-inflation parameter is at the boundary of the parameter space. Unfortunately, this somewhat technical detail really distorts the upper quantiles in Wilson's brief simulation study. There is a convenient workaround where the ZIP model family is expanded to include zero-deflated models too, but I don't know if this has been implemented here or not.

https://stats.stackexchange.com/questions/182020/zero-inflated-poisson-regression-vuong-test-raw-aic-or-bic-corrected-results https://wlv.openrepository.com/bitstream/handle/2436/622610/Vuong1.pdf?sequence=2

bvieth commented 4 years ago

Hello,

I consider the Poisson and Zero-Inflated Poisson Distribution OR the Negative Binomial and Zero-Inflated Negative Binomial to be non-nested and therefore use the Vuong Test to compare the distributional model fits. In addition, I consider the Negative Binomial and Poisson distribution to be nested and thus use the likelihood ratio test to compare the distributional model fits. There is no model fit comparison of Negative Binomial with Zero-Inflated Poisson or Poisson with Zero-Inflated Negative Binomial.

You can find more information concerning evaluation metrics and comparisons in the supplements of the original powsimR publication.

I hope I was able to answer your questions.

Kind regards Beate

ekernf01 commented 4 years ago

Yes, you did! Thank you!