Biases

[Jorgeferrrus]

Dear Dr. Roth,

I am currently using your package to compute the unconditional and conditional biases after passing a pretest with different data. I understand that the unconditional bias is equivalent to "deltatrue." However, I am uncertain about the conditional bias. Should it be directly interpreted as "meanafterpretesting," or is there an additional calculation involved?

Thank you for your time.

Best regards,

Jorge

[jonathandroth]

If I understand your email correctly, I think the answer is yes.

In the notation of the paper, `deltatrue' is \delta, and meanAfterPretesting is E[ \betahat | \betahat_{pre} \in B], where B is the set of "insignificant coefficients"

Best, J

On Mon, Aug 12, 2024 at 5:46 AM Jorgeferrrus @.***> wrote:

Dear Dr. Roth,

I am currently using your package to compute the unconditional and conditional biases after passing a pretest with different data. I understand that the unconditional bias is equivalent to "deltatrue." However, I am uncertain about the conditional bias. Should it be directly interpreted as "meanafterpretesting," or is there an additional calculation involved?

Thank you for your time.

Best regards,

Jorge

— Reply to this email directly, view it on GitHub https://github.com/jonathandroth/pretrends/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6EXFDUQFXXMHTS2SWLHWDZRCAAHAVCNFSM6AAAAABMLZ4FHSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3DANJRGYZTOMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Jorgeferrrus]

Dear Dr Roth,

Thank you for your quick response!

I’m still having some trouble understanding the function's notation. In Figure 1 in the paper, the conditional bias is expressed as E[\tauhat − \tau^* | \betahat_{pre} \in B]. Based on my understanding, wouldn't the conditional bias in figure 1 be computed (with the results of the pretrends function) as:

\frac{1}{M} \sum meanAfterPretesting_post - \frac{1}{M} \sum \betahat_post ? (1)

I tried to replicate the results with one of the 12 papers you surveyed; however the result is much closer to the one in the paper if I don't substract \frac{1}{M} \sum \betahat_post to \frac{1}{M} \sum meanAfterPretesting_post, i.e. if I set the conditional bias equal to average of meanAfterPretesting_post. This makes me think that the approach (1) is wrong.

*** I refer to "meanAfterPretesting_post" and "\betahat_post" as the subvectors after treatment.

Best,

Jorge

[jonathandroth]

The conditional bias as you said is:

E[\tauhat − \tau^* | \betahat_{pre} \in B]

The pretrends function considers the case where \tau^* = 0 (everything comes from the pre-trend, so there is no treatment effect). So then the bias is just E[ \tauhat | \betahat \in B] = meanAfterPretesting.

On Mon, Aug 12, 2024 at 10:39 AM Jorgeferrrus @.***> wrote:

Dear Dr Roth,

Thank you for your quick response!

I’m still having some trouble understanding the function's notation. In Figure 1 in the paper, the conditional bias is expressed as E[\tauhat − \tau^* | \betahat_{pre} \in B]. Based on my understanding, wouldn't the conditional bias in figure 1 be computed (with the results of the pretrends function) as:

\frac{1}{M} \sum meanAfterPretesting_post - \frac{1}{M} \sum \betahat_post ? (1)

I tried to replicate the results with one of the 12 papers you surveyed; however the result is much closer to the one in the paper if I don't substract \frac{1}{M} \sum \betahat_post to \frac{1}{M} \sum meanAfterPretesting_post, i.e. if I set the conditional bias equal to average of meanAfterPretesting_post. This makes me think that the approach (1) is wrong.

*** I refer to "meanAfterPretesting_post" and "\betahat_post" as the subvectors after treatment.

Best,

Jorge

— Reply to this email directly, view it on GitHub https://github.com/jonathandroth/pretrends/issues/7#issuecomment-2284168409, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6EXFHISBLLWJ2FGQMM7WDZRDCLHAVCNFSM6AAAAABMLZ4FHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBUGE3DQNBQHE . You are receiving this because you commented.Message ID: @.***>