Solomon four-group designs

[ASKurz]

Please leave suggestions for studies using a Solomon four-group design.

[ASKurz]

Consider Galassi et al (1974; https://psycnet.apa.org/doi/10.1037/h0037094) Assertive training in groups using video feedback. Here's the abstract:

Investigated the effectiveness of group assertive training with 16 male and 16 female college students who were nonassertive (as measured by the College Self-Expression Scale, Galassi et al, 1974). Ss were assigned randomly to 2 experimental and 2 control groups. Experimental Ss received 8 training sessions consisting of videotape modeling; behavior rehearsal; video, peer, and trainer feedback; bibliotherapy; homework assignments; trainer exhortation; and peer-group support. All Ss were posttaped enacting role-playing situations. Significant differences were found between experimental and control Ss on the College Self-Expression Scale, the Subjective Unit of Disturbance Scale, eye contact, length of scene, and assertive content, but not on response latency. Significant pretest effects were evident for several of the behavioral dependent variables.

With only 8 persons to each group, this naturally leads into a discussion of power.

[ASKurz]

Consider the Study 4 from Lai et al (2014; https://doi.org/10.1037/a0036260), Reducing implicit racial preferences: I. A comparative investigation of 17 interventions. Here's the abstract:

[Correction Notice: An Erratum for this article was reported in Vol 143(4) of Journal of Experimental Psychology: General (see record 2014-29995-001). The Methods section did not mention an exploratory measure that was included in Study 4 but was not analyzed. The information is provided. The article also includes discrepancies in the ranking of interventions between the first two paragraphs in General Discussion and Figure 1. Figure 1 was correct; the General Discussion was not. Revised text provides the corrected rankings and analyses to include data from Study 4.] Many methods for reducing implicit prejudice have been identified, but little is known about their relative effectiveness. We held a research contest to experimentally compare interventions for reducing the expression of implicit racial prejudice. Teams submitted 17 interventions that were tested an average of 3.70 times each in 4 studies (total N = 17,021), with rules for revising interventions between studies. Eight of 17 interventions were effective at reducing implicit preferences for Whites compared with Blacks, particularly ones that provided experience with counterstereotypical exemplars, used evaluative conditioning methods, and provided strategies to override biases. The other 9 interventions were ineffective, particularly ones that engaged participants with others’ perspectives, asked participants to consider egalitarian values, or induced a positive emotion. The most potent interventions were ones that invoked high self-involvement or linked Black people with positivity and White people with negativity. No intervention consistently reduced explicit racial preferences. Furthermore, intervention effectiveness only weakly extended to implicit preferences for Asians and Hispanics.

After filtering based on the authors' exclusion criteria, this is a well-powered N = 5,000 study with about 200 persons per cell. They used a Solomon "four"-group design in quotes because the authors actually had a 13 X 2 factorial design with 13 experimental conditions, each of which did or did not have a pre-test. The authors have a large study repository on the OSF at https://osf.io/lw9e8/ and you can download their files at https://osf.io/lw9e8/files/. This is a nice opportunity to fit a multilevel model with condition as the pooled grouping factor. In this way, you look at the interaction effect typical of the Solomon four-group design in the form of the posterior SD for the difference in the pre-test dummy variable.

This is also a nice study in that the criterion variable is an IAT d-score. Though it would be great to analyze the data using a within- and between-person model, the data on the OSF have already been aggregated such that each participant's IAT performance is summarized by d. Thus, this would lead into a good discussion about whether to analyze data on aggregates and would point to the chapter on within-person factorial designs for reaction-time data.

Since this is an example of a group-level study with a large number of conditions, this is also a candidate to consider under issue https://github.com/ASKurz/Experimental-design-and-the-GLMM/issues/9.

Note: Initial data wrangling and modeling code live in the Lai et al (2014) folder on Dropbox. Also, you can find a twitter thread on how to visualize the difference in differences plot here.

[ASKurz]

Consider Coyne et al (2022; https://doi.org/10.1007/s10461-022-03710-9), Understanding and HIV stigma in response to undetectable = untransmittable messages: Findings from a nationally representative sample in the United Kingdom. Here's the abstract:

'Undetectable = Untransmittable', or 'U = U', is a message which communicates the scientific consensus that people living with HIV who maintain an undetectable viral load cannot sexually transmit HIV to others. This research aimed to empirically test whether a protection-framed U = U message is more effective at decreasing HIV stigma and increasing perceived accuracy of U = U than a risk-framed message. A nationally representative UK sample ($N = 707$) completed an online experiment. Participants viewed one of two U = U messages (protection-framed or risk-framed) and completed an online questionnaire. No evidence of a difference in HIV stigma at post-test or in perceived accuracy of U = U was found between the two message frame conditions. A minority of participants were aware of U = U prior to participation. Post-intervention, the majority of participants rated U = U as at least somewhat accurate. Higher understanding of U = U was associated with lower post-test stigma following a protection-framed message. Following a brief intervention, among a sample predominantly unaware of U = U previously, there was an overall favourable rating of U = U. No evidence was found for an effect of message framing on HIV stigma or perceived accuracy of U = U, but participants who completed a pre-test measure of stigma rated U = U as less accurate.

Coyne and colleagues randomly assigned $N = 707$ UK adults into one of four conditions, based on the typical $2 \times 2$ Solomon four-group design for an online study aimed at reducing HIV stigma. Their primary outcome measure was a measure stigmatising attitudes towards persons living with HIV. The measure was composed of 6 5-point Likert-type items taken from a broader survey. These items had not been previously examined from a psychometric perspective. Given their distributions and the distribution of the sum score, this is a nice opportunity to practice the multilevel cumulative probit IRT-type approach.

Also, the authors provided their raw data files and scripts on the OSF at https://osf.io/rbq7a/.