Direction of Bias

[dholford]

Hello,

I have a follow up question from the review video related to the direction of bias. In the review guide the direction of bias based on the sign was summarized here:

However, in the test score / class size example we actually have a negative bias (at least I think it's -1.483). We only end up with a positive number when we calculate the magnitude of the bias (dividing by a negative and thus ending up with a positive 51%):

So when we're talking about direction of bias and the sign of the bias, is it about the sign of the bias or the sign of the magnitude of the bias?

The test score example is clearly an instance where adding the omitted variable (teacher quality) changes the conclusion about the "program" (class size), which is described in the first picture under "If bias is positive." But the bias is in fact negative, it's the magnitude of the bias that is positive.

Again any guidance would be much appreciated!

Best, Dylan

[kaseympotts]

I think (huge emphasis on think) that the sign of the bias matters for direction in the first instance because it's that point in the process where we are trying to determine IF we are over-estimating or underestimating effect.

Then in the second part, the sign of positive or negative doesn't tell us direction in the magnitude, because at that point it's telling us we if overestimated or underestimated by that amount. Positive= overestimated; negative=underestimated.

So, if size of bias in the second example had been a -51%, that would mean we under-estimated. Since it's positive, it tells us we overestimated.

[Schlinkert]

Good question, Dylan. And good answer, Kasey. The sign (+ or =) of the bias is what will make the program look better of worse. The magnitude is by how much.

[dholford]

That was my understanding, but that's where I get confused with the class size and test score example. In that instance we have a negative bias (-1.483), but then according to the second image in my original post having a negative bias "the program will only look better if we add the omitted variable." But adding SES reduced the magnitude of the impact of class size from -4 to -2 (which I'm interpreting as making the program look worse)? Or I guess it made it look better in that it was less negative, but I thought when talking about magnitude the absolute value is what mattered because positive or negative the impact is less??

[BrettMFoster]

IMHO, -2 seems far better than -4 because it's showing a less biased result after adding the control; in fact, 51% less biased. The "magnitude" helps justifies adding controls to the model... whether we know the control already (SES) or if there is an omitted variable or unknown third variable .