dschoenig
commented
3 years ago In the course of working on issue #11, I decided to remove the (dubious) comparison based on adjusted R-squared for non-nested models (of which some don't even satisfy the assumptions of a linear model) that also have a diverging number of observations. Adjusted R-squared is discussed only as a measure of fit.
That still leaves the question of how to (and whether) to discuss model comparison in this workshop.
In the examples presented, model comparison is mostly performed via adjusted R². The most common, alternative ways for comparison should be mentioned, such as AIC (which currently appears to be the most widely used in ecology) and comparing nested models via ANOVA. AIC (to find the model with the lowest reduces predictive error) and R² (to find the best-fit model) can both form part of a criterion-based approach, while using ANOVA in this case is more a hypothesis test approach (which is okay for comparison but can be problematic for selection).
More on that can be found, for example in, chapter 10 of Faraway, J. J. (2014). Linear models with R. CRC press.
Additional clarification is also needed for model selection: throwing out predictors based on the results of t-tests shown in the summary can be a problem (especially for running analyses with many models). The concept of parsimony is stated in a way that suggests that one always wants the model with the fewest predictors, and it is even used as justification for step-wise model selection (which is a very controversial practice). This can be seen as a misinterpretation of the principle. In practice, using and interpreting the "full" model (with predictors chosen according to theory and hypotheses) is recommended in many cases. See also: Whittingham, M. J., Stephens, P. A., Bradbury, R. B., & Freckleton, R. P. (2006). Why do we still use stepwise modelling in ecology and behaviour?. Journal of animal ecology, 75(5), 1182-1189.