Does order of treatment/control group matter for esc_mean_gain function?

[Mihaylova1]

Hi Daniel & all,

Does anyone know whether group1 should always be treatment group and group2 as control? For some functions like convert_d2r and esc_B, it is stipulated to enter experimental group values as group 1 and control group as group 2. However, for functions like esc_mean_sd and esc_mean_gain, this is not stipulated.

I've been using the function esc_mean_gain entering treatment group as group1 and getting opposite results from what is anticipated. Just wondering whether there is a necessary order for these two functions and what computations they are doing to obtain the result?

Thank you for your input

[strengejacke]

Could you provide a small reproducible example (maybe also with desired results/output)?

[Mihaylova1]

Hi Daniel,

Thanks a lot for getting back to me so quickly!

I made the below example to outline the issue. In the “test”, we can see that group 1 (experimental) should have a positive effect size as it has a definite increase from pre to post test with all other values constant. Hedges’ g, however, is negative:

library(esc)
test <- esc_mean_gain(
  pre1mean = 7.56, pre1sd = 3.2 , post1mean = 24.65, post1sd = 3.5, grp1n = 20, 
  pre2mean = 7.5, pre2sd = 3.5, post2mean = 7.9, post2sd = 3.6, grp2n = 20,
  es.type = "g"
)
test
#> 
#> Effect Size Calculation for Meta Analysis
#> 
#>      Conversion: mean gain score to effect size Hedges' g
#>     Effect Size:  -4.7371
#>  Standard Error:   0.8321
#>        Variance:   0.6924
#>        Lower CI:  -6.3680
#>        Upper CI:  -3.1062
#>          Weight:   1.4442

When I reverse the two groups (putting experimental as group 2 and control as group 1), I get the anticipated positive effect

library(esc)
test_rev <- esc_mean_gain(
  pre1mean = 7.5, pre1sd = 3.5 , post1mean = 7.9, post1sd = 3.6, grp1n = 20,
  pre2mean = 7.56, pre2sd = 3.2, post2mean = 24.65, post2sd = 4.5, grp2n = 20,
  es.type = "g"
)
test_rev
#> 
#> Effect Size Calculation for Meta Analysis
#> 
#>      Conversion: mean gain score to effect size Hedges' g
#>     Effect Size:   4.3837
#>  Standard Error:   0.8099
#>        Variance:   0.6560
#>        Lower CI:   2.7963
#>        Upper CI:   5.9711
#>          Weight:   1.5245

It seems counterintuitive to enter group 1 as control when it is stipulated in other functions in the package that group 1 is treatment. How does the function compute the effect size exactly? Does the treatment group have to be entered as group 2?

It would be great if you could clarify and provide some insight.

Thank you, Mariela

[Mihaylova1]

Hi Daniel,

I am just following up on my below email. Have you had a chance to go through the code and function?

Look forward to hearing back from you!

Thank you, Mariela

[strengejacke]

Not yet, but I'll look into this the next days.

[Mihaylova1]

It would be great if I could get feedback on this by the end of the week.

Thank you, Mariela

[Mihaylova1]

Hi Daniel,

Just checking in on this. Have you had a chance to look into this yet?

Thanks in advance, Mariela

From: Mariela Mihaylova Sent: Thursday, 13 February 2020 09:39 To: strengejacke/esc reply@reply.github.com Subject: RE: [strengejacke/esc] Does order of treatment/control group matter for esc_mean_gain function? (#6)

It would be great if I could get feedback on this by the end of the week.

Thank you, Mariela

From: Daniel notifications@github.com<mailto:notifications@github.com> Sent: Thursday, 13 February 2020 09:30 To: strengejacke/esc esc@noreply.github.com<mailto:esc@noreply.github.com> Cc: Mariela Mihaylova mariela.mihaylova@unidistance.ch<mailto:mariela.mihaylova@unidistance.ch>; Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: Re: [strengejacke/esc] Does order of treatment/control group matter for esc_mean_gain function? (#6)

Not yet, but I'll look into this the next days.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/strengejacke/esc/issues/6?email_source=notifications&email_token=AOPMB6HFUHI67GFX75NECE3RCUAI3A5CNFSM4KTU6HAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELT3HEY#issuecomment-585610131, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOPMB6AJICI4GMFOO7RSVKLRCUAI3ANCNFSM4KTU6HAA.

[poschfeld]

Hi @strengejacke,

following up on this question in the context of transforming an F-Test using esc_f. Obviously, the output includes a positive effect size and CI, although from reading the original study we know it should be negative. Is there a way to automatically label the esc_f calculation for direction, without manually inverting ES and CI values?

Many thanks for the awesome package btw, found it through the Harrer et al. (2021) guide and really appreciate how straight forward it is!

All the best, Poul

[linneagandhi]

Hi @strengejacke and @Mihaylova1 I'm also having issues with this function esc_mean_gain -- looking into the source code, I believe that there may be an error:

See lines 161-163 compute mean gain scores for groups 1 and 2 if (missing(gain1mean)) gain1mean <- pre1mean - post1mean if (missing(gain2mean)) gain2mean <- pre2mean - post2mean

I would have expected this to read: post-pre not pre-post. That would explain why @Mihaylova1 your results seemed flipped (and why my own seem flipped as well in using the code today :-)

If the developer team is busy and @Mihaylova1 you are still working on the project, you could just reverse the sign manually or edit the source code that is locally stored on your computer. (Assuming my diagnosis is right of course!) Happy to be corrected if I'm missing something! ~Linnea Gandhi