Closed Alanapiereid closed 2 years ago
Hi, @Alanapiereid!
The docstring of get_feature_importance
method says that for multiclass case it returns ndarray with shape (n_objects, classes_count, n_features + 1). The last value for each class is bias (i.e. feature-independent value).
Thank you, that's helpful. Do you know a way of getting importance values for each word? Again, I would usually just run an sklearn-compatible classifier (like out-of-the-box CatBoostClassifier() ) + a vectorizer through SHAP or LIME using a Pipeline object, but this custom class seems incompatible there due to the way the data has been wrapped in the Pool() class.
CatBoost doesn't support of getting importance for each word, only for whole text feature.
Problem: SHAP Values not interpretable catboost version: 1.0.4. Operating System: Linux CPU: 4 vCPUs, GPU: None
I have built a sentiment analysis classifier using a custom class derived from CatBoostClassifier(). I want to get feature (word) importance weights from my model, in a format such as
-0.098432 violent
-0.063453 terrible
I want these values for each word in a string that is unseen by the model (not training data, so no label). I want to do this on a per-text basis.
I have built a multi-label classifier (with 3 possible labels). Because the model is a custom class object (not out-of-the-box CatBoostClassifier), which is then saved and then loaded, it does not have the native attributes/methods of the CatBoostClassifier class. To that end I wrote the get_feature_importance function below for my custom CatBoostPipe class:
When constructing my model class, I structured the train/val/test sets using Pool() as below:
Here is what happens when I create an instance of my custom class, then use the get_feature_importance method that I have overwritten:
I get that 1 is the number of data samples, 3 the number of classes. But what are the two features for each class? Could this be a tokenization issue? I was expecting an array of weights per word as that is what I have got before for SHAP values from other models (i.e. out of the box CatBoostClassifier + TfIDF Vectorizer fed into a sklearn Pipeline object).
Here is the full class: