Lakens / TOSTER

TOST equivalence test and power functions
35 stars 18 forks source link

TOSTpaired.raw significance discrepancy #25

Closed ontogenerator closed 6 years ago

ontogenerator commented 6 years ago

Dear developers,

I am currently making a shiny App for power analysis and experimental design assistance, based primarily on your TOSTER (v0.3) R package. When working on examples with repeated measures I ran into an inconsistency between the graphical output of the TOSTpaired.raw function and the significance of the reported TOST results.

For instance, in this example:

TOSTpaired.raw(n = 10, m1 = 24, m2 = 22, sd1 = 2, sd2 = 2, r12 = 0.5,
               low_eqbound = -3, high_eqbound = 3, alpha = 0.05)

the 90% CI overlaps with the SESOI boundary, yet the resulting p values are both significant, with the overall result of the TOST being significant (i. e. equivalent).

After looking deeper into the code, I may have found the reason for this behavior.

The current version of the code for the t value calculation is (e.g. for t1):

t1 <- ((m1_s - m2_s) - (low_eqbound * sdif))/se

However, according to my interpretation of section 1.3 in Mara & Cribbie (2012), the formula should be:

t1 <- ((m1_s - m2_s) - low_eqbound)/se

Of course, I may have overlooked something, but it would be nice to know either way.

Cheers,

Vladislav

Lakens commented 6 years ago

Hi, I'm traveling but it looks like we did not remove the multiplication by sdiff when creating to raw function from the standardized function. Thanks for finding this! We will update it. Can you email me with info about the shiny app? Thanks!


From: ontogenerator notifications@github.com Sent: Thursday, April 5, 2018 4:06:10 PM To: Lakens/TOSTER Cc: Subscribed Subject: [Lakens/TOSTER] TOSTpaired.raw significance discrepancy (#25)

Dear developers,

I am currently making a shiny App for power analysis and experimental design assistance, based primarily on your TOSTER (v0.3) R package. When working on examples with repeated measures I ran into an inconsistency between the graphical output of the TOSTpaired.raw function and the significance of the reported TOST results.

For instance, in this example:

TOSTpaired.raw(n = 10, m1 = 24, m2 = 22, sd1 = 2, sd2 = 2, r12 = 0.5, low_eqbound = -3, high_eqbound = 3, alpha = 0.05)

the 90% CI overlaps with the SESOI boundary, yet the resulting p values are both significant, with the overall result of the TOST being significant (i. e. equivalent).

After looking deeper into the code, I may have found the reason for this behavior.

The current version of the code for the t value calculation is (e.g. for t1):

t1 <- ((m1_s - m2_s) - (low_eqbound * sdif))/se

However, according to my interpretation of section 1.3 in Mara & Cribbie (2012), the formula should be:

t1 <- ((m1_s - m2_s) - low_eqbound)/se

Of course, I may have overlooked something, but it would be nice to know either way.

Cheers,

Vladislav

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FLakens%2FTOSTER%2Fissues%2F25&data=02%7C01%7CD.Lakens%40tue.nl%7C6005e0b2c5bc4898ae3a08d59b06d077%7Ccc7df24760ce4a0f9d75704cf60efc64%7C1%7C0%7C636585375926748009&sdata=z6oYa2WDDISdQR9hfRWslQr4GshDP3%2BgAUFsV52NiB0%3D&reserved=0, or mute the threadhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGMH4RgR94H4TzewcTDua4CuSJYTts3Uks5tljLigaJpZM4TIn9j&data=02%7C01%7CD.Lakens%40tue.nl%7C6005e0b2c5bc4898ae3a08d59b06d077%7Ccc7df24760ce4a0f9d75704cf60efc64%7C1%7C0%7C636585375926748009&sdata=%2FxKBQhDFGuWOxvjQnUGZU96uoATLWk50HgStOvxDbTI%3D&reserved=0.

Lakens commented 6 years ago

Hi, I fixed the code, and submitted a new version (3.1) to CRAN. The version on GitHub is corrected. Thanks for informing me - you rock!