section 6.4.1: misstatement of posterior

[poldrack]

the text says "p-values don’t tell us the probability of the data under any given alternative hypothesis that we might be interested in – that’s the posterior probability P(H_1|data)". don't you mean to say "p-values don't tell us the probability of a specific alternative hypothesis given the data - that's the posterior probability"? I might reframe this paragraph to make the point a bit more crisply that p-values are not probabilities of hypotheses: they are the probability of the data given one specific hypothesis (the null).

[poldrack]

I'm also not so sure about this in the following paragraph: "That is, a failure to reject the null does not mean that you can accept the null." This sort of implies that there is equipoise between the null and alternative prior to the test, and one is choosing between them. Whereas in reality, one starts the NHST process by assuming the null and then trying to falsify it. So, when p > .05, one does in some sense "accept the null", but only because they assumed it was true from the beginning (as required by NHST). Perhaps a crisper way to say it is to be clear up front that a null results does not provide any additional evidence for the null.

[mcfrank]

I pushed some minor changes here. "any given" -> "specific"

I think the point of taking the stance of the null being true is a bit subtle though... I tried to clarify with a few text edits but my primary priority was to keep students from making the naive "p > .05 means null is true" inference, which I do view as a mistake.