Support for more adjustment

[phliulei]

Thank you for your great package.

Some paper recommended Dunnett’s C, Dunnett’s T3, Games-Howell, and Tamhane’s T2 for post hoc adjustment. Because

When the assumptions of equal sample sizes and variances were violated, only four tests adequately maintained the Type I error rate: Dunnett’s C, Dunnett’s T3, Games-Howell, and Tamhane’s T2.

Can emmeans support for (one of) them?

[rvlenth]

emmeans already supports all of these, at least to a close approximation. However, you need to understand that emmeans does not choose what model to use, it only computes results based on a model that you fitted. So if you fit a model using lm() or aov or afex::aov_ez() or a number of others, the underlying assumptuion of all those models is that the error variance is homogeneous across treatments.

So, the first step in understanding this is to study and understand what is said in the FAQ vignette qabout Welch's tests -- see https://cran.r-project.org/web/packages/emmeans/vignettes/FAQs.html#nowelch -- which explains how to fit a model using nlme::gls() that accommodates unequal variances. The rest of what I have to say assumes you have done that. Under that condition, ...

• Games-Howell Use adjust = "tukey"
• Tamhane T2: Use adjust = "sidak"
• Dunnett C or Dunnett T3: Use adjust = "mvt". However, Dunnett's C is based on an additional assumption that you have limited the number of tests to m comparisons; so to gain the advantage of his C method, you have to omit some of the pairwise comparisons. For instance, if you are comparing 5 means, there are 54/2 = 10 pairwise comparisons. If you test all 10 with adjust = "mvt", that's Dunnett's T3. If you choose a smaller number of them, say only 6, and use adjust = "mvt", then that's Dunnett's C. However, you have to choose those 6 comparisons without* looking at the observed differences or related statistics (pre-planned comparisons). To do otherwise would be cheating and it would violate the integrity of the method.

[phliulei]

Thank you so much. But I'm still a little confused:

When the assumptions of equal sample sizes (not the variances) were violated, adjust = "tukey" can be used taking Games-Howell's place in emmeans? As the papers said, tukey is not fit for unbalanced design, so we use Games-Howell .

[rvlenth]

As I tried to explain earlier, it depends on the model that you are giving to emmeans()..

• If you used a model that presumes homogeneous variances, and use adjust = "tukey" in emmeans(), that is the Tukey procedure we all know and love.
• If you use a model that allows for unequal variances and give that to emmeans() with adjust = "tukey", that is the Games-Howell procedure. In other words, Games and Howell adapted the Tukey method by using the Tukey critical values applied to the unequal-variances t statistics.

The above are two different models. That's why the second one is not the regular Tukey method.

[phliulei]

I got it! Thank you for you patience.