aobo-y
commented
10 months ago @Luckick Sorry for the late reply! This is great question.
But unfortunately, this is a known limitation of Captum right now. Captum assumes you provides a single baseline whose values are used to replace part of the features. We are aware this is not perfect for 2 use cases:
1st, Shapley may need to sample multiple baseline instances to approximate the distribution the part of "missing" features and the function value should be the expectation of all of them.
2nd, the features may not be independent. So the baselines for the "missing" features depends on the existing unchanged features.
A single baseline cannot support these 2 requirements. We are discussing of supporting pass baseline argument as a function, (input, perturbation_mask) -> baselines, so users can customize how the baselines are sampled. cc @vivekmig @NarineK
Hi,
From my understanding, when shap synthesize the dataset, we need to replace part of the features, according to the distribution of such feature from a background dataset (training set or samples from training set). Thus the shap package requires to provide background dataset for the explainer.
However, it seems captum does not need the background dataset (the baseline could be a single instance or have to have same size with test set). How does captum synthesize the dataset for permutation?
Thanks!