clarifying when approximations are used by `effectsize`

[IndrajeetPatil]

Since report calls effectsize on model objects to compute effect sizes, effectsize often needs to use an approximation in case insight::get_data() doesn't work.

I think the report output should somehow note this for the user, lest they get confused as to why the values don't match with other package outputs (including effectsize).

For example, https://github.com/easystats/report/issues/183#issuecomment-858420376

[mattansb]

effectsize gives a warning, but this is suppressed by report... :/

[DominiqueMakowski]

If not suppressed, can it be muted?

[bwiernik]

We should use tryCatch() to catch the warning and then adjust the report output accordingly.

[leighclark]

If I might be so bold as to make a suggestion: if an error message does get generated in the future, it might be useful to display the "correct" format for getting the "right" results as part of the message, as @mattansb posted in the other thread.

[IndrajeetPatil]

Thinking out loud here:

How about-

• effectsize outputs get a new attribute called approximate with a default value of FALSE
• when insight::get_data() fails, and an approximation is used, this attribute value will change to TRUE
• when report sees that this attribute is TRUE, it will add "(approximate)" to whatever effect size is being reported

For example, here it would change to

Cohen's d (approximate) = 2.03, 95% CI [1.13, 2.91]

[leighclark]

For example, here it would change to

Cohen's d (approximate) = 2.03, 95% CI [1.13, 2.91]

That would be great, and maybe (instead of what I suggested above) it could describe how to "correct" the approximation in the docs or in an example or something :)

[mattansb]

how to "correct" the approximation in the docs or in an example or something :)

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

[leighclark]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

You are operating on the assumption that novices like me know what "formula notation" is ;)

[bwiernik]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

Or better, don't use htest functions, just fit a linear model 😜

[mattansb]

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

The formula notation for *.test() functions is when you use a formula to let the function know the structure of you data (:

t.test(mpg ~ am, data = mtcars) # formula

t.test(mtcars$mpg[mtcars$am==0], mtcars$mpg[mtcars$am==1]) # pass values

[bwiernik]

But really @leighclark Don't use the *.test functions. Use lm() instead. https://lindeloev.github.io/tests-as-linear/

[mattansb]

Hmm this is good advice maybe for inference in general - but report will treat it like a linear model (so no Cohen's d...)

[leighclark]

But really @leighclark Don't use the *.test functions. Use lm() instead. https://lindeloev.github.io/tests-as-linear/

hey @bwiernik I think (know) we are at totally different stages of our psychology and statistics careers, so I am still at the stage where I have to do the things that are given to me, but thanks for the tip and I will take a look at it!

Cheers, Leigh

[bwiernik]

@mattansb https://github.com/easystats/effectsize/issues/351

[LukasWallrich]

This would be a great addition to the docs / examples ... I just needed to find this issue to figure out how to report t.tests

how to "correct" the approximation in the docs or in an example or something :)

This would essentially be "don't use formula notation for htest functions" - that's the solution! (:

[mattansb]

Results from effectsize() should have an approximate attribute that can be used.