Meaning of 3 and 4 asterisks in the distinguishing statatements are NOT in the manual

[bfsc]

On the manual it says that "Differences that are significant at a p-value < .05 are indicated with '*', and differences significant at a p-value < .01 are indicated with '**'"

However, there are situations where there are THREE AND EVEN FOUR ASTERISKS. What is the meaning of them?

Below is an example:

[aiorazabala]

Dear Bruno @bfsc, I'm very sorry for the late reply. I've been replying this question by email, but missed your comment here. The response is below, and I'll add this matter to update the help pages in the next update.

You can find it exactly in the raw code of the qdc function (in the package files, here: .../qmethod/R/qdc.R). I adapt it here:

p >= 0.05 <- "" (i.e. nothing)
p < 0.05 <- "*"
p < 0.01 <- "**"
p < 0.001 <- "***"
p < 0.000001 <- "****"

The p<0.1 isn't common in Q (see Brown 1980, p284). As you see, it's p < x, rather than p ≤ x (although this shouldn't make much of a difference). The stars follow the custom you mention, except for the label of four stars (****). There is no strong reason for the last one to be that way, and I should change that for consistency actually. The reason was that, in some of my tests, the p values were coming up very very low and I wanted to distinguish which ones.

[aiorazabala]

Added the full information to the help page for qdc() (in the next release 1.5.5), and changed the last one (four stars) to become "6*" to make it more logical.

I included the testing for p < 0.000001 while testing, but consider that there's no need to test for such small p-values, given it's not usual in the method to do so (PQMethod doesn't test beyond p<0.1)