rvlenth
commented
1 month ago It is important to understand the distinction between the object and what you see when you display it. The famous quote of Rene Magritte suits this situation: "This is not a pipe" (it is a picture of a pipe).
The object created by emmeans, contrast, and several other functions is of class emmGrid (I think this is what you meant, not EM-mean). These objects have the information needed to compute estimates, but not the estimates themselves. When you display an emmGrid object, you can see what happens by looking at its print method:
> emmeans:::print.emmGrid
function(x, ..., export = FALSE)
print(summary.emmGrid(x, ...), export = export)
<bytecode: 0x000002393f8ce488>
<environment: namespace:emmeans>Note that when you print the object, it calls summary(). And if you print it again, it calls summary() again. It is summary.emmGrid that computes the estimates, adjusted P values, etc. Since the mvt method uses a randomized algorithm, we get different results each time summary() is called.
If you do something like this:
emm <- emmeans(..., adjust = "mvt")
semm <- summary(emm)
emm # line 3
emm
emm
semm # line 6
semm
semmyou will get slightly different results from lines 3, 4, and 5, but the same results from lines 6, 7, and 8. That's because semm is of class summary_emm which is an annotated data frame. It has the computed estimates, adjusted P values, etc.
The other side of this coin is that you can do, say, contrast(emm, "pairwise") and get results, but contrast(semm, "pairwise") will give you an error because semm is just a data frame.
There is a lot of documentation in the emmeans package; Honestly, so much it is overwhelmong. Look at ? "emmGrid-class" for direct documentation of the class. There is also information in vignettes, e.g., a section in the "basics" vignette that explains these objects.
It would be inappropriate to add a seed argument to emmeans because that function has nothing to do with P-value adjustments. It's possible to add such an argument to summary.emmGrid, but I think that adds unneeded complexity. All you need is to know that the results of summary.emmGrid() are immutable. Also, ideally, from a computing standpoint, it is undesirable to repeatedly recompute the same quantities, as will happen when you repeatedly display an emmGrid object even if there is no "mvt" adjustment in force.
adrianolszewski
Hello, I assume the described issue comes from the fact that EM-mean is an object with methods performing some operations multiple times whenever the result is called.
I just realized that despite of setting the seed before the adjustment (Monte Carlo integration in mvtnorm), the results are still inconsistent between my printout from R console and the graph where I used it to display p-values.
I just "called" the name of the resulting variable multiple times in different places after it was assigned - and... it jumped. This costed me a lot of analyses to be repeated with putting the emmeans result explicitly into a data.frame... I would like to warn others, especially if they have tens of analyses to be reported.
I understand the behaviour, but it's really easy to forget about it. I mean - not about the fact that mvt needs the seed to be set. but that despite that it can jump anyway.
and then I just call the result:
OK, the difference is just little, but if this was something on the boundary of rounding, the issue would be much more visible.
Is there maybe any way to "lock" the result? Or is the only way to avoid that casting it into a data.frame explicitly or setting the seed any time I refer to this variable?
Is this very behaviour mentioned explicitly somewhere in the documentation? If not, I think it really should.
Or - maybe you could add the "seed" parameter to emmeans, so it's always honoured every time "mvt" is requested without the need to set it "externally" not only every time one calls the update/contrast, but also calls the assigned variable too?