pp. 176-177

[roberto-viviani]

Chapter "Demonstration of adding pure noise predictors to a model"

• p.176: "Unsurprisingly, ... residual sigma estimate has slightly increased." At the rounding of the printout, median sigma and MAD_SD are identical for the two models.
• The text continues with comparison of LOO R2 and LOO predictive log scores (p. 176). However, there is no code or printout that demonstrates how these estimates are obtained from the fits. Instead, these computations are introduced on p. 177, but the text on p. 177 does not refer to the previous estimates: "having fit the model, we use the loo to compute the leave-one-out cross validation log score: loo_3 <- loo(fit_3)" However, fit_3 was never defined; instead, p. 176 has just introduced fit_3n. It is easy to miss the 'n' and think that loo_3 was computed from fit_3n. This does not appear to be the case, however; reading carefully and comparing, one notices that the elpd_loo of loo_3 was reported on p. 176 as the predictive score of an unnamed model (-1876.1). There is no way to understand this other than fitting the unnamed model oneself, and check that it is fit_3 by computing the elpd_loo (because on p. 176 there is no printout of the computations of -1876.1). Furthermore, since fit_3n has noise predictors and the title of the chapter refers to noise, one is easily misled to thinking that all the comparisons that follow concern the fit_3n model with noise, with the result that they make little sense. They concern the unnamed model instead, which is fit_3 and has no noise predictors. I also suggest to keep the same name for what elpd_loo computes, instead of alternately referring to 'predictive log score', 'estimated log score', 'leave-one-out cross validation log score' etc.

[avehtari]

Thanks!

I removed the "residual sigma estimate has slightly increased".

The folder for the code was mentioned on p. 156 when the KidIQ was first time used in this chapter. I now added the footnote about the code folder also on p. 176. The book often has less code than the online code appendix, which has the code reproduce all computations.

I added on p. 177 a reference to p. 156 which introduced fit_3.

Thanks also on the feedback about the log score. I went through the section and did not make any changes. In all cases they are log scores, but depending on the context log score as associated with term emphasising certain property of that log score and these were not meant to be different names of the quantity. I'll recheck this in a few days if I'll then see better how this could be clarified.

Fixes will appear in the updated pdf soon.