Identification of optimal policy with multiple treatments

[JFrischknecht]

Dear Michael - Thanks again for these great lecture notes and all the updates you made in the last months :)

I have now revisited chapter 9: Policy Learning. I was a bit puzzled by a step in the identification proof on slides 28/29, maybe you can help me out here:

On slide 28 you define

$$\Gamma(w,X):= m(w,X) + \frac{D(w)(Y - m(w,X))}{e_w(x)}$$

And on slide 29 we arrive at

$$\pi^* = arg max{\pi} \quad E\left[\sum{w = 0}^{T}\ \mathbb{1}[\pi(X) = w] \ E[Y(w) | X ] \right]$$

We know that

$$E[Y(w) | X ] = E\left[m(w,X) + \frac{D(w)(Y - m(w,X))}{e_w(x)} | X\right] = E[ \Gamma(w,X) | X] $$

However, here I think the definition of $\Gamma(w,X)$ is getting a bit confusing: If I understand this correctly, the score

$$m(w,X) + \frac{D(w)(Y - m(w,X)}{e_w(x)}$$

for a fixed $w$ is a function of both $Y$ and $X$, and not only a function of $X$. So instead we should probably write $\Gamma(w,X, Y)$. And then it should be clear that it does not hold in general that

$$E[ \Gamma(w,X,Y) | X] = \Gamma(w,X,Y)$$

since there is still randomness in $Y$ given $X$.

But it seems that on the last line of slide 29 (equation 8), you replace $E[Y(w) | X ]$ with $\Gamma(w,X)$. Instead I think on the last line it should say $E[ \Gamma(w,X,Y) | X]$ instead.

In any case, for the identification result it does not really matter, I think. Identification is already achieved on this line, since we know that $E[Y(w) | X ]$ is identified through the data $(Y_i, D_i, X_i)$

$$\pi^* = arg max{\pi} \quad E\left[\sum{w = 0}^{T}\ \mathbb{1}[\pi(X) = w] \ E[Y(w) | X ] \right]$$

and thus the overall expression is just a functional of the observed distribution $(Y_i, D_i, X_i)$ and hence identified, right?

[MCKnaus]

As always, you are just on point. Sorry for the confusion. I have uploaded an update. Does this resolve the confusion?

Thank you :-)

[JFrischknecht]

Yes this clarifies my question - Thanks! Already looking forward to reading the new slidedeck on Online Learning :)