Non-parametric ttest2 iteration exceeding max possible iterations

[hwrlea001]

Hi

I am trying to perform a 1d, non-parametric, 2 sided t-test, and I am getting the error: 'Number of specified iterations (1000) exceeds the maximum possible number of iterations (924)'

I have 2 groups of data vectors (6 observations per group), being compared. I have tried shortening the vectors (from 2000 data points to 25 data points per vector), but am still always being limited to exactly 924 iterations. If I change the specified iteration number to 924 the code runs fine.

What causes this iteration max? And is it OK to just set the 1000 to 924, or is this too few iterations?

Thanks Lee

[0todd0000]

Permutation tests like this and others in spm1d work as follows: Imagine that you stack your six observations for the two groups on top of each other so that the 12 observations can be labeled as AAAAAABBBBBB. This 'original' set of labels will produce a certain t value. Then randomly permute the labels (e.g. ABAAAAAABBBBB) and re-compute the t-value. Repeating for many random permutations produces a distribution of t-values, and this distribution is the distribution from which inferences are drawn (i.e., p-value calculations).

The maximum number of permutations is determined by the number of observations, not the number of domain points (e.g. time points). For a two-sample test the total number of unique label permutations is N! / (NA! NB!) where NA and NB are the numbers of observations in the two groups, and where N = NA + NB. So in your case NA=NB=6, N=12, and 12! / (6! 6!) = 924. Thus there are only 924 unique ways of permuting the labels: AAAAAABBBBBB.

If you try to force 1000 permutations an error is generated because there are fewer than 1000 unique permutations.

Incidentally, if the data are normally distributed then this permutation-based t distribution will approach the Student's t distribution and will converge to it precisely for an infinitely large sample. The distribution is also symmetric when all permutations are executed because opposite labels like AAAAAABBBBBB and BBBBBBAAAAAA produce equal values with opposite signs.