mcfrank
commented
1 year ago great comments, tried to address these... the learning gradient one is tough, but I put a bit of front matter to specify that people can skip some of the formal bits and go straight to the worked example....
I'm working through the book sequentially and these are my comments on chapter 7. I've also made some direct edits to the text for more minor things and writing adjustments and issued a pull request.
"For example, a simple statistical model might assume that a series of coin tosses were generated via a weighted coin. The goal of estimation in this context is to assess the most likely weight of the coin given the data. This model can then be used to make inferences about whether the coin's weight differs from some null model (a fair coin, perhaps)." [NB - this is my reworded reversion submitted via pull request]. <- isn't this the wrong way around? Wouldn't we first want to know whether the coin is weighted? And then, if it is, proceed to estimation?
"What we’ll learn to do in this chapter is to make a model of this situation that allows us to reason about the magnitude of the milk-order effect while also estimating variation due to different people, orders, and tea types." <- I think this is clear; but it would be cool to ground it in the concepts readers learned earlier in Chapter 1 — Experiments. Like how would we represent the contribution of these factors in a DAG?
"Regression models" <- shall we say what the word regression means? I think sometimes these concepts can sound intimidating and mysterious until you hear the rationale behind them.
"independent variables are sometimes called predictor variables, covariates, or features" <- we should clarify perhaps, that the reverse is not true (you can't call those things independent variables because they are not being manipulated)?
I like the more descriptive subscripts here, but they are different to the ones we used in Chp 5 — change the ones in Chp5?
I like the sidenote starting "Here’s a quick reminder that “model” here is a way of saying “set of assumptions about the data generating procedure.” I wonder if we should move it to chapter 6 where we first mention this idea, somewhat mysteriously: "First, we assume a model for how the data were generated".
"Conventionally, regression models are written with “Beta’’ symbols" <- can we clarify how this relates to our previous use of Beta to describe the causal effect? And perhaps worth clarifying whether we are talking about population parameters here or sample statistics.
If there's a learning gradient to this book I feel like it suddenly got much steeper in this chapter! — sure its harder material so that's somewhat inevitable, but I do think in previous chapters we had more text dedicated to building intuitions, providing examples, and unpacking concepts. By contrast, in this chapter there's a lot of assumed prior knowledge and description without much explanation. I also wonder if we could ground everything more in what readers learned earlier, e.g., DAGs, and keep reminding how this more complex stuff is helping to achieve the reader's goals.
"The addition of these crossed random intercepts of participants and items would begin to address the challenge posed by Clark (1973) in our case study above" <- I can't see a case study anywhere in this chapter!
Figure 7.3 — data points, labels, etc are quite small
"a control condition in which the puppet had eaten too much peanut butter and couldn’t talk". Dev psych control conditions are the cutest :)
"Any given model is likely wrong" <- it might not be obvious to readers that we mean reasonably accurate but strictly wrong (rather than erroneous, outright wrong). Perhaps we should say suboptimal?