search

FBartos / zcurve

zcurve R package for assessing the reliability and trustworthiness of published literature with the z-curve method
https://fbartos.github.io/zcurve
10 stars 1 forks source link

add observed discovery rate to output #3

Closed Lakens closed 3 years ago

Lakens commented 4 years ago

Thanks for this very useful and well-made package.

The plot gives us the observed discovery rate, and a 95% CI - but as far as I can see, these numbers are not available in the results outputted by z-curve (not even when all = TRUE). To report results in R Markdown, it would be useful to have these numbers in the output of the z-curve function (maybe including the number of studies included in the analysis, and the number of significant results, also in the plot but not in the output.

FBartos commented 4 years ago

Hi Daniel,

I'm glad that you like the package. I added information about the observed discovery rate (ODR) and the number of used, significant, and supplied z-statistics / p-values. It's located under the model convergence message for now. I'm still thinking about including ODR next to the estimates, however, it's not dependent on the estimated model (and it would break the output table if bootstrapping was disabled because the ODR CI is computed analytically.)

I will push this version to CRAN soon to make it directly available.

Cheers, František

fit <- zcurve(rnorm(50,3)) summary(fit)

Call: zcurve(z = rnorm(50, 3))

model: EM via EM

Estimate l.CI u.CI ERR 0.859 0.726 0.940 EDR 0.858 0.222 0.940

Model converged in 29 + 73 iterations Fitted using 41 z-values. 50 supplied, 41 significant (ODR = 0.82, 95% CI [0.68, 0.91]). Q = -45.58, 95% CI[-49.23, -40.52]

https://github.com/FBartos/zcurve/commit/1ceb2ae2aebadfc25bdfd781900cb3afba3d2336

Lakens commented 4 years ago

Thanks - however, this does not solve the issue I raised - I would like to be able to get the value of ODR and report it in the text -so, z_res$fit$ODR would allow my to just get the number 0.82, and use it in a completely reproducible RMarkdown file. No rush, but in principle it is always useful to allow people to extract any calculated number from the stored result.

FBartos commented 4 years ago

Oh, now I understand what you are after. Yeah, that's true. I added convenience functions for accessing all of the estimates (so you don't have to go directly into the z-curve object).

They either print the values into the console ERR(fit)

Estimate l.CI u.CI 0.753 0.522 0.893

with a possibility to select formating EDR(fit, 2)

Estimate l.CI u.CI 0.53 0.21 0.89

or directly access the raw values (using [["Estimate"]] / [["l.CI"]] / [["u.CI"]] or $Estimate / ..., apart from the significant_n and included_n which store the single value within [["n"]].

ODR(fit)[["Estimate"]]

0.82

ODR(fit)[["l.CI"]]

0.6807861

You can check ?estimates.zcurve for all the functions: ERR(object, round.coef = 3) EDR(object, round.coef = 3) ODR(object, round.coef = 3) Soric(object, round.coef = 3) file_drawer_ration(object, round.coef = 3) expected_n(object, round.coef = 0) missing_n(object, round.coef = 0) significant_n(object) included_n(object)

Hope that this addresses the issue. https://github.com/FBartos/zcurve/commit/957745f1f71848c24c99e489383d96dad75f9057

FBartos commented 3 years ago

1.0.6 - version including this functionality is now on CRAN