Part VI: Model - "Confirmatory" text

[saghirb]

Issue to consider

In the Part VI: Model it is stated

You can use an observation as many times as you like for exploration, but you can only use it once for confirmation. As soon as you use an observation twice, you’ve switched from confirmation to exploration.

I don't feel that this is not strictly correct as by design you could use some observations twice as part of a confirmatory analysis. For example, in a clinical trial, by design you could have a placebo (control) group, low active dose and high active dose where you want to test for efficacy for both active doses versus the control group. Here the testing procedure to control for multiplicity could first test the high dose versus placebo followed by low dose versus (a form a hierarchical testing). The second test would only be conducted if the first test passes some preset criteria which would be mean that the placebo observations would have been used twice for confirmation.

As a note the ICH E9 statistical guidelines for the pharmaceutical industry state:

A confirmatory trial is an adequately controlled trial in which the hypotheses are stated in advance and evaluated.

Some paraphrasing of this text might be a better option.

[hadley]

I think that's a sufficiently subtle point that it's beyond the scope of this book.