Shapley values with dependent features

[martinju]

Hi, and thanks for an amazing book!

From my point of view, there is an at best imprecise statement related to the problems with dependent features for Shapley values, which may lead to confusion for the reader. From Section 5.9.5 Disadvantages [of Shapley values]:

...Another adaptation is conditional sampling: Features are sampled conditional on the features that are already in the team. While conditional sampling fixes the issue of unrealistic data points, a new issue is introduced: The resulting values are no longer the Shapley values to our game, since they violate the symmetry axiom, as found out by Sundararajan et. al (2019) and further discussed by Janzing et. al (2020).

My view on this: An explanation game is defined with a characteristic function/contribution function (called val_x(S) in this book). Thus, using either (A) the marginal expectation val_x(S) = E[f(X)|do(x_S)] = E[f(x_S,X_Sbar)] (also called the interventional conditional expectation), or (B) the proper conditional expectation: val_x(S) = E[f(X)|x_S] = E[f(x_S,X_Sbar)|x_S] (also called the observational conditional expectation), simply defines different games. Thus both of these Shapley value games provide perfectly valid Shapley values obeying all the Shapley value properties. [Above, we let x = (x_S,x_Sbar) where x_S is the subvector of x corresponding to S, and x_Sbar is the remaining part of x.]

It is not clear to me whether the description in the book is restricted to any of the two definitions above. In fact, the second equation of 5.9.3.1 specifies that val_x(S) is indeed an expectation, but since the mathbb P is not defined (maybe this is on purpose???), it could be any of the two. Hence, it think it is incorrect to write "The resulting values are no longer the Shapley values to our game, since they violate the symmetry axiom". I am just guessing here, but this quote may have appear as a result of some rather vague statements in the Janzing et. al (2020)-paper which is referred to here. However, reading that paper carefully, in particular the section called "The problem with the symmetry axiom" they never claim that Shapley value symmetry axiom (which is defined with respect to val_x(S)) is violated with the explanation game in (B). They rather argue that (B) violates a property they define in the start of the paper relating to the symmetry of explanations directly on f -- which is not the same as the Shapley value symmetry axiom.

Instead of referring to the rather one-sided Janzing et al (2020) and preceeding Sundararajan et. al (2019) papers, I would recommend the more balanced paper by Chen et. al (2020) https://arxiv.org/abs/2006.16234 entitled "True to the model or true to the data?". They argue that sometimes explanation game (A) is preferrable and sometimes explanation game (B) is preferrable -- it all depends on the type of problem and application one seeks to explain.

Bottom line: I believe the book would benefit from a minor revision in terms of this topic, at least I woud recommend modifying the sentence "The resulting values are no longer the Shapley values to our game, since they violate the symmetry axiom..." as that is not correct and is likely to leave the reader with the wrong impression.

I hope you will take this issue into consideration, and will be happy to discuss this rather cumbersome issue further :-)

[davidrosenberg]

I actually came here to ask about the exact same thing, although now the book claims that the conditional distribution would violate the "Shapley axiom of Dummy".

[christophM]

@martinju thanks for that detailed discussion.

I agree with your view. I made a PR, and would be happy if you think it is a fair assessment now. Feel free to comment on the pull request, or make one yourself.

[martinju]

Great to hear! I will take a look at the PR and comment/commit directly there within a few days 👍