search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
59 stars 29 forks source link

Enhancement requests: Implementation of Artifact corrections for Meta-Analysis #115

Open fplatz opened 6 years ago

fplatz commented 6 years ago

Template for enhancement requests

* Enhancement: Meta-Analysis * Purpose: I'm really excited that JASP' functionality has been increased for being able to conduct meta-analyses. Due to my experiences with my last meta-analysis I wondered whether future versions would be able to correct correlations (or differences as well) for attenuation due do low reliability as mentioned by Schmidt & Hunter (2014). If it would possible to enter the "raw" values (either mean, sd or the studies' effect sizes), then it would be great to have an opportunity to enter values for the reliabilites of the measurements. To the best of my knowledge it is not possible even for commercial software such as CMA to correct the observed correlation (or difference) to get the uncorrected so called theoretical effect sizes as well as its disattenuated CI. That would be a major improvement for conducting meta-analysis. However, I'm really impressed about your great work and I'm looking for the next version of JASP! Best wishes, Friedrich
AlexanderLyNL commented 6 years ago

Hi Friedrich,

Thanks for the encouraging work. It'll take a bit more time to do this if there's no package available for this. If you know one that suits this purpose we're happy to look into it. The meta-analysis module is not done yet and future releases will include new features.

Cheers, Alexander

fplatz commented 6 years ago

Hi Alexander, thanks for that quick reply! Perhaps Martin Viechtbauer's blog might be an interesting starting point. I think that the example might give an impression of how the disattenuation could be implemented in his package or in JASP, which uses metafor already. Perhaps it might be an interesting collaboration for Viechtbauer working with you resulting in an update of his package. Additionally this issue is on his to-do list too as well as Bayesian meta-analysis.

Cheers, Friedrich

TarandeepKang commented 5 months ago

Hi All,

I wonder if,- for the purpose of flexibility,- we should consider switching away from the current implementation of the Hunter-Schmidt method which uses Metafor? The ability to use the requested corrections is not implemented in that package yet. But there is much greater ease of use in psychmeta:

Dahlke, J. A., & Wiernik, B. M. (2019). psychmeta: An R Package for Psychometric Meta-Analysis. Applied Psychological Measurement, 43(5), 415–416. https://doi.org/10.1177/0146621618795933 Hunter J. E., Schmidt F. L., Le H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594–612. Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Sage. doi: 10.4135/9781483398105. Wiernik, B.M. & Dahlke, J.A. (2020) Obtaining Unbiased Results in Meta-Analysis: The Importance of Correcting for Statistical Artifacts. Advances in Methods and Practices in Psychological Science. 3(1): 94-123

TarandeepKang commented 4 months ago

Hi @tomtomme I'm wondering if you can perform your usual magic and bring this to the attention of the right person? I don't think the usual person responsible for meta-analysis has been tagged here? I think this feature is important, and as you can see there hasn't been a response in several years. Vielen dank im voraus! :)

Best,

Tarandeep

EJWagenmakers commented 4 months ago

Rumor has it that Frantisek is working on improving the meta-analysis module...

fplatz commented 4 months ago

Cool! Great to hear this! Am 24.06.2024 um 18:13 schrieb EJ @.***>: Rumor has it that Frantisek is working on improving the meta-analysis module...

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

TarandeepKang commented 4 months ago

Let's hope that rumour proves to be true…

tomtomme commented 4 months ago

@TarandeepKang I am happy to help and maybe I can give some helpful context here:

If you wonder about the current maintainers of each module, see: https://github.com/jasp-stats/jasp-issues/blob/master/.github/scripts/keywords.js This file results in a "github action" that auto-selects the current/correct maintainer to new issues, if the requesters selects the right module. From time to time those maintainerships change of course, but from the file history of the file you can see that it is kept up to date regularily. So maintainerships of old issues like this can get outdated and then I will try to correct this. But also: From time to time a dev feels like it and takes over an issue that he is not officially the current maintainer of.

To conclude: If you find further instances of issues that seem to have the wrong maintainer, that is usually only a problem, if the maintainer of the package has completely moved onto other ventures and is completely missing from the above file.

Cheers, Thomas

FBartos commented 4 months ago

Hi all,

Great suggestions and I will take a look at psychmeta. It's very unlikely that we will incorporate it in the following release i.e., 0.20, but there are some good news.

0.20 will feature:

(the updates allows for setting custom weights, so with a bit of additional calculation in JASP you can do the same analysis as here: https://www.metafor-project.org/doku.php/tips:hunter_schmidt_method?s%5B%5D=artifact#artifact_corrections )

I'm giving a talk about the updates at a conference tomorrow, so you can have a sneak peak at the upcoming functionalities in these two videos:

https://github.com/jasp-stats/jasp-issues/assets/38475991/49449be2-6c24-4d9c-8327-1f1a095ebcec

https://github.com/jasp-stats/jasp-issues/assets/38475991/c92da6c6-846c-4bc1-a2ec-f9e23585dcfb

fplatz commented 4 months ago

Wow! That sounds very interesting! Happy to see, that MA has received such a high priority! Fantastic! Especially the multi-level MA is of high importance! Looking forward to see it running! And again: Thank you very much! Am 25.06.2024 um 15:34 schrieb František Bartoš @.***>: Hi all, Great suggestions and I will take a look at psychmeta. It's very unlikely that we will incorporate it in the following release i.e., 0.20, but there are some good news. 0.20 will feature:

newly implemented effect size calculation, utilizing all features of the escalc function in metafor reimplemented simple meta-analysis, that will greatly improve meta-regression, clustering, forest plots, bubble plots, advanced optimizer settings etc newly implemented multilevel/multivariate meta-analysis that allows for specifying wide range of random effect structures

I'm giving a talk about the updates at a conference tomorrow, so you can have a sneak peak at the upcoming functionalities in these two videos: https://github.com/jasp-stats/jasp-issues/assets/38475991/49449be2-6c24-4d9c-8327-1f1a095ebcec https://github.com/jasp-stats/jasp-issues/assets/38475991/c92da6c6-846c-4bc1-a2ec-f9e23585dcfb

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>