Ex2.9 The beta can only be a function of training data. Your chosen beta is also a function of testing data

[coolcty]

Ex2.9 The beta can only be a function of training data. Your chosen beta is also a function of testing data.

[YuhangZhou88]

Hello,

"The beta can only be a function of training data." Could you elaborate more? Thanks

[coolcty]

Hello,

"The beta can only be a function of training data." Could you elaborate more? Thanks

When we do the least square optimasation, we choose a beta with only the informaion of the traing data. And the beta is determined if the training data is determined. The beta hat is only optimal among this kind of beta.

But your chosen beta depends also on the testing data, giving it an extra randomness, and is out of this range. Then the inequality (1) may not be satisfied.

[YuhangZhou88]

For inequality (1),

• the first "=" is by definition of R_{tr} (\hat\beta)
• the "<=" is by definition of \hat\beta (when training data x and y are fixed, \hat\beta is therefore fixed, and the inequality holds for any \beta )
• the last "=" is from IID assumption

[coolcty]

The inequality does not hold for any \beta, but only those beta that are \sigma(training data) measurable.

[YuhangZhou88]

for any \beta that is \sigma{X, Y}-measurable

[coolcty]

for any \beta that is \sigma{X, Y}-measurable

If you use X, Y to denote the N pairs of training data, yes. And the testing data is independant of the training data, so is not \sigma{X, Y}-measurable.