Isabella's suggestions - Githubissues

Some was already addressed, so picking up from a relevant point: See also #59.

Continuing on with a fresh mind:

The statement “The take-home message is that we can interpret our result in a linear or nonlinear space, but it can be a bit difficult.” seems to beg for a bit of elaboration.

For binary or binomial regression models, we have 3 possible scales of interpretation; each scale comes with its own metric(s).

As I see it, the scales are:

Log Odds

Odds

Probability

The reader should know that we can easily go from one scale to another.

The scale-specific metrics can be:

Log Odds

Odds; Odds Ratios

Probabilities; Differences in Probabilities; Ratios of Probabilities; etc.

(I hope I got this right. 😂)

The model is fitted on the first scale but that is the least interpretable to a lay audience; so results can be expressed on the higher-level scale to achieve increased interpretability.

This is nitpicking now BUT non-linearity comes into play when we focus on EFFECTS of predictors on the odds or probability scale.

In particular, for continuous predictors, effects that were postulated as linear on the log odds scale become non-linear on the probability scale.

That non-linearity means they can no longer be described with a single number (i.e., a slope) - unless one ventures into marginal effects territory. They have to be visualized and described qualitatively, and/or described numerically where they change shape.

Also, predictors which do not interact explicitly in their effects on the log odds scale can end up interacting implicitly on the probability scale.

So that using different scales can lead to conflicting messages (which is fine - that’s just the way these models work).

Someone here on Twitter (forget whom) described this as: each scale tells a different story and the stories can be different.

In this sense, models with non-linear links are multi-story models.

The reader gets to choose what stories to tell depending on their audience.

Because this multi-story business is such a tricky thing in applied statistical practice, I think it deserves more elaboration than the original sentence hints at.

Of course, I may be biased here, but that’s only because I myself have struggled for years until this all clicked.

My pet peeve is jumping into a model’s R formula before explaining what the model is trying to do conceptually. 🤣

From this viewpoint, when introducing the first logistic regression model in the GLM chapter (on good vs bad movie ratings), this is needed:

a) A clarification for the reader that there are two flavours of logistic regression - binary and binomial. (Data format dictates which one we choose, etc.)

b) An explicit mention of the Bernoulli distribution as a special case of the Binomial distribution for n = 1.

c) An explanation on how the first logistic regression model is in fact a binary logistic regression model; base R does not have a bernoulli family so we will still fit it with the binomial family. (Other packages like brms do have a bernoulli family.)

d) A comment on the “purpose” of the model: description / explanation AND/OR prediction. (I am using ‘description’ for focusing on reporting associational effects; ‘explanation’ for reporting causal effects).

How we present model results is driven by the purpose of the model.

For d), we can state for instance that we will focus on investigating the (associational?) effects of word count (i.e., review length?) and gender of reviewer on the probability/chance of a good movie review, etc.

“We need to know what those results mean.”

We do - but at this stage, we have to be crystal clear on the purpose of the modelling, the modelling scale on which we will report the results, the metric of interest for that scale, etc.

We need to be prepared upfront!

Your figures are a thing of beauty!!!

For “Figure 5.4: Model Predictions for Word Count Feature”, I would suggest plotting the curves for both genders on the log odds scale (left) and on the probability scale (right).

Showing this augmented figure BEFORE interpreting the model summary would clarify so many things for the reader. They would see first hand how the linear effect of word count for each gender becomes on the log-odds scale becomes non-linear on the probability scale.

Why keep saying “it’s difficult to interpret this effect”? Just show the connection between scales visually.

These are generally very good points and need to be considered. Here are some thoughts on what we might incorporate or not:

Some of this gets into weeds, and as this isn't a stats book nor specifically geared toward academia, such details are probably best left to textbooks that have that focus. However, I think the recent changes, some of which are a result of these and previous suggestions, have shored up some of the gaps.
I/we personally still don't find odds or their ratios easily interpretable, and this is particularly prevalent with clients in consulting across many disciplines. When it comes to prediction, the focus is exclusively on probabilities and/or the classes, and that's where we feel most of the interpretation lies as well.
On the 'nitpicking' part, I don't think it is nitpicking as we actually cover this, and we even have a call out box about it.
Some people use logistic regression with no intention of interpreting the specific coefficients and related statistical output, and that's okay. Or more specifically, they are only concerned with predicted probabilities/classes. This suggests after defining the feature set, jumping into the model is okay (assuming the data is sound in the first place).
a-c: We mention the binomial/count aspect but do not focus on it because we're specifically talking about a classification model setting for demonstration, which is also extremely more common than the binomial count model. We talk about the bernoulli special case in both a callout and footnote.
d: It might be good to clarify to some extent, but we are making no assumption about how one might want to use the model here or elsewhere. The purposes noted are not mutually exclusive and typically done together (which we talk about in another chapter), and nothing requires one to focus on one of those purposes exclusive to others at any point in the modeling process.
d cont'd: We also have a whole chapter that focuses on causal issues in modeling that attempt to outline the issues there. It's also probably not necessary to point out with every model that we aren't doing a causal model, as that is the default. I think it may be helpful to prop up the intro a bit though, here and possibly elsewhere.
"How we present model results is driven by the purpose of the model." This is spot-on in a general sense, and we have explicitly said as much in an earlier chapter. It's important to also know that many users of these models are unconcerned with the explanatory aspect of a model or its statistical result. Even then, they may need to explain results in some way to various stakeholders, who often will not understand the statistical aspect at all.
I do think it'd be good to show the log-odds linear word-count relationship, but there isn't an explicit interaction of it with gender in this model, so breaking the effect out by gender would possibly confuse. I think we started to talk about the implicit interactions when we are on the probability scale at one point, but thought that got into more detail than we wanted.
"Why keep saying “it’s difficult to interpret this effect? Just show the connection between scales visually." This is exactly the point we try to make and why we aren't as focused on statistical interpretation of the coefficients. One can run a GLM without interpreting anything statistical about it.

Again these are all great points to consider in some fashion. We'll think more on how to incorporate some of these issues without getting bogged down in a discussion of things that are exclusive to academic reporting expectations (in some domains).

m-clark / book-of-models

Isabella's suggestions #88