ewvanzwet
commented
10 months ago Hi Yefeng,
Since the distribution of the z-values is a mixture of normals, it follows that the distribution of the SNRs is also a mixture of normals (just subtract 1 from the variances of the components). Moreover, the conditional distribution of the SNR given the observed z-value is also a mixture of normals. The formulas are in the appendix of the paper with Schwab and Senn.
The z-value is just the SNR plus standard normal error. So, the conditional distribution of the z-value of an exact replication study (z_repl) given the observed z-value of the original study (z_orig) is equal to the conditional distribution of the SNR given the observed z-value of the original study plus standard normal error. So that's also a mixture of normals.
Now if you want to condition only on the statistical significance of the original study (not z_orig itself), then you have to integrate this distribution over the conditional distribution of the z_orig given |z_orig|>1.96. For the probability of a significant replication, given the statistical significance of the original study you get
P(|z_repl| > 1.96 | |z_orig| > 1.96) = integral_z_orig P(|z_repl| > 1.96 | z_orig > 1.96) f(z_orig | |z_orig| > 1.96) d z_orig
I do this integration over f(z_orig | |z_orig| > 1.96) by simply averaging over the observed z-values which are larger than 1.96 in absolute value. To write all this out in formulas is "tedious" (as mathematicians would say) and probably not that helpful for our intended audience.
I think it's quite clear that my calculations are correct, because the two methods (Monte Carlo and exact) give almost exactly the same result.
Hope this helps! Erik
From: Yefeng @.> Sent: vrijdag 27 oktober 2023 1:46 To: Yefeng0920/replication_EcoEvo_git @.> Cc: Zwet, E.W. van (MSTAT) @.>; Mention @.> Subject: [Yefeng0920/replication_EcoEvo_git] mathematical formula (Issue #3)
This GitHub issue is used for the mathematical formula involved in the paper. This might need input from @ewvanzwethttps://github.com/ewvanzwet
I briefly explain my purpose. There are two ways of computing the parameter of interest. The first is based on the simulated data and uses the definition of the parameter of interest to compute it. For example, the following is the definition formula of replication rate:
$$ z \times z{\text{repl}} > 0 \ \ \text{and}\ \ |z{\text{repl}}| > 1.96$$
The corresponding code is: snr=rmix(10^5,p=p,m=m,s=sqrt(sigma^2 - 1)) z.orig=snr + rnorm(10^5) # original z.repl = snr + rnorm(10^5) # replication replicate=(z.orig * z.repl > 0) & (abs(z.repl) > 1.96) mean(replicate) # unconditional probability of replication mean(replicate[abs(z.orig)>1.96]) # conditional probability of replication given |z|>1.96
The second is based on each parameter of interest's conditional distribution of the real data (rather than simulation). For example, we can use the following code to estimate the conditional probability of replication:
z vs. replication z=sort(abs(d$z)) replicate=rep(NA,length(z)) s=sqrt(sigma^2-1) for (i in 1:length(z)){ post=posterior(z[i],p,m,s) pp=post$p pm=post$pm ps=post$ps replicate[i]=1 - pmix(1.96,p=pp,m=pm,s=sqrt(ps^2 +1)) }
The formula relevant to the first type of estimation method is quite clear to me. I would appreciate if @ewvanzwethttps://github.com/ewvanzwet would like to help derive the formula relevant to the second type of estimation method.
- Reply to this email directly, view it on GitHubhttps://github.com/Yefeng0920/replication_EcoEvo_git/issues/3, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKTA2C7PR76RHKAHWIW5ASTYBLY23AVCNFSM6AAAAAA6R7LH36VHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3DINJTGA3TEMQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
Yefeng0920
itchyshin
This GitHub issue is used for the mathematical formula involved in the paper. This might need input from @ewvanzwet
I briefly explain my purpose. There are two ways of computing the parameter of interest. The first is based on the simulated data and uses the definition of the parameter of interest to compute it. For example, the following is the definition formula of replication rate:
$$ z \times z{\text{repl}} > 0 \ \ \text{and}\ \ |z{\text{repl}}| > 1.96$$
The corresponding code is:
snr=rmix(10^5,p=p,m=m,s=sqrt(sigma^2 - 1)) z.orig=snr + rnorm(10^5) # original z.repl = snr + rnorm(10^5) # replication replicate=(z.orig * z.repl > 0) & (abs(z.repl) > 1.96) mean(replicate) # unconditional probability of replication mean(replicate[abs(z.orig)>1.96]) # conditional probability of replication given |z|>1.96The second is based on each parameter of interest's conditional distribution of the real data (rather than simulation). For example, we can use the following code to estimate the conditional probability of replication:
# z vs. replication z=sort(abs(d$z)) replicate=rep(NA,length(z)) s=sqrt(sigma^2-1) for (i in 1:length(z)){ post=posterior(z[i],p,m,s) pp=post$p pm=post$pm ps=post$ps replicate[i]=1 - pmix(1.96,p=pp,m=pm,s=sqrt(ps^2 +1)) }The formula relevant to the first type of estimation method is quite clear to me. I would appreciate if @ewvanzwet would like to help derive the formula relevant to the second type of estimation method.