Number of required observations per participant/condition etc

I was wondering is there an advised number of observations to be included per subject, and condition and/or time point if running ANOVA's)

As far as I know there is no consensus regarding the number of per-subject observations. I suspect that, as the number of subjects grows, the number of per-subject observations has an increasingly small effect on the results. Since inter-subject variance tends to be much larger than intra-subject variance, then it is quite possible that one observation per subject is sufficient. This is just a guess, and would need to be verified with real datasets.

Is the advised number of included observations different between a t-test analysis compared to ANOVA or 2Way Repeated Measures ANOVA?

Yes, in general. Given a specific hypothesized effect size, the more complex the design, the more subjects are required achieve a specific power. For example: whereas 12 subjects may be sufficient for a t test to detect an effect size of 0.5 with a power of 0.8, several more subjects will likely be needed to detect the same effect at the same power level in a two-way ANOVA.

Presuming that these tests all have multiple subjects and that the goal is population inference, then actually there is effectively just one observation per subject.

In routines where a SUBJECT variables is not an explicit requirement (e.g. spm1d.stats.ttest2, spm1d.stats.ttest_paired, spm1d.stats.anova1, etc.) it is possible to submit 5 observations per subject, for example, but to the routine it would look like the number of subjects is five-times larger than it actually is, which would be highly problematic.

Does adding more observations than the recommended number improve the analysis or is it meaningless?

Adding more per-subject observations to spm1d's current RM-ANOVA procedures serves primarily to stabilize the estimates of (a) within-subject means and (b) smoothness. If the data are well-behaved then adding more observations is likely to be inconsequential; essentially the same results are expected if you pre-calculate within-subject means.

As an illustration, consider the script below which is a modification of the ex_anova2rm.py example script. The main change is that a second dataset has been created, consisting of five random observations per-subject and per-condition, instead of just the single observation that is included in the original dataset.

import numpy as np
import matplotlib.pyplot as plt
import spm1d
import rft1d

# load data:
#    This dataset consists of one simulated observation
#    per subject and per AB-combination
dataset    = spm1d.data.uv1d.anova2rm.SPM1D_ANOVA2RM_2x2()
Y,A,B,SUBJ = dataset.get_data()
fwhm       = 15   # true smoothness of the simulated data

# create five-observations per-subject, per AB-combination
np.random.seed(0)
Y1,A1,B1,SUBJ1 = [], [], [], []
n          = 5
for i,(y,a,b,s) in enumerate(zip(Y,A,B,SUBJ)):
    yy     = y + 0.1 * rft1d.randn1d(n, y.size, fwhm)
    Y1.append( yy )
    A1    += [a]*n
    B1    += [b]*n
    SUBJ1 += [s]*n
    # Set the original single observation to the 5-observation mean
    # (This is like pre-computing the subject mean for each AB
    #   combination)
    Y[i]   = yy.mean(axis=0)
Y1    = np.vstack(Y1)
A1    = np.array(A1)
B1    = np.array(B1)
SUBJ1 = np.array(SUBJ1)

# check dataset sizes:
print('Variable sizes (original): ', Y.shape, A.shape, B.shape, SUBJ.shape)
print('Variable sizes (5 per-subject): ', Y1.shape, A1.shape, B1.shape, SUBJ1.shape)

# run ANOVA:
F     = spm1d.stats.anova2rm(Y, A, B, SUBJ, equal_var=True).inference(0.05)
F1    = spm1d.stats.anova2rm(Y1, A1, B1, SUBJ1, equal_var=True).inference(0.05)
# consider just the Main A effects for brevity:
f     = F[0]
f1    = F1[0]

# critical threshods:
print('\n\nSmoothness estimates:  %.3f, %.3f' %(f.fwhm, f1.fwhm))
print('\n\nCritical thresholds for Main A effect:  %.3f, %.3f' %(f.zstar, f1.zstar))

# plot results:
plt.close('all')
fig,(ax0,ax1) = plt.subplots( 1, 2, figsize=(8,3), tight_layout=True )
f.plot( ax=ax0 )
f.plot_threshold_label(ax=ax0)
f1.plot( ax=ax1 )
f1.plot_threshold_label(ax=ax1)
ax0.set_title('Only pre-calculated means')
ax1.set_title('With 5 observations')
plt.show()

The printed outputs are:

Variable sizes (original):  (28, 101) (28,) (28,) (28,)
Variable sizes (5 per-subject):  (140, 101) (140,) (140,) (140,)

Smoothness estimates:  14.993, 14.979

Critical thresholds for Main A effect:  27.142, 27.154

And here is the figure:

fig

You'll see that there are minor numerical differences in the smoothness estimates, and that this causes a small change in the critical threshold. Practically this effect is likely to be negligible unless the dataset has odd characteristics.

If you are concerned about how many observations to include I suggest going through the exercise in the script above with your own data, and comparing just the pre-calculated mean case to the all-observation case. In most cases I'd expect there to be negligible differences.

E.g. If the recommended observations to enter into SPM at each time point/condition was 5 and you had 12 jumps collected at that time point would it be better to include all of them?

In general it is best to include all data unless you find a problem with a specific observation (e.g. an outlier). Including more observations tends to stabilize mean and smoothness estimates.

0todd0000 / spm1d

Number of required observations per participant/condition etc #261