It may be good to have in 5.5.5 a counter-point to permutation-based feature importance metrics. In this paper, Hooker and Mentch argue that permutation-based metrics are brittle because permutation forces models to extrapolate. This problem gets worse for correlated features because the effective dimension of the data manifold is smaller, but in high dimensions the effective dimension of the data manifold is almost always much smaller than the number of features, so simply checking for pairwise correlations may not be sufficient to overcome this problem.
Very nice work on this book!
It may be good to have in 5.5.5 a counter-point to permutation-based feature importance metrics. In this paper, Hooker and Mentch argue that permutation-based metrics are brittle because permutation forces models to extrapolate. This problem gets worse for correlated features because the effective dimension of the data manifold is smaller, but in high dimensions the effective dimension of the data manifold is almost always much smaller than the number of features, so simply checking for pairwise correlations may not be sufficient to overcome this problem.