The explain() gives a maximum of only 9 features for explanation

[sourabhsparkala]

Hello,

I have a regression dataset with 174 features, and 10 observations(samples).

explanation <- explain(dataset_test, explainer, n_features=20, feature_select="lasso_path")

I am trying to get the model explain atleast 20 features for each observation, Hence the resultant dataset should have 20 features (20 rows) and its respective features_value for each case.

Here If I have 10 observations (as the dataset is regression). Then the resultant "explain dataset" should have 20*10 rows. But instead the # of features to explain the model seems to be restrictive to count 9. Even after I have mentioned the n_features=20 as the parameter.

Is my understanding correct. Or is this a bug?

Please let me know.

Regards Sourabh

[thomasp85]

Are you sure this is not just the print method only showing the top rows? If not, can I get you to provide a reproducible example...

[thomasp85]

rereading your issue, it doesn't seem to be a printing misunderstanding - sorry. Can you provide a reprex and I'll be happy to look into it

[sourabhsparkala]

@thomasp85

Thank you for the post.

I started digging in the lime code.

I guess it was exhibiting only 9 top features because the dataset I was experimenting with, had many features with standard deviation = 0.

Apologies for the misunderstanding.

Thanks and Regards Sourabh

[sourabhsparkala]

I understand that the features with standard deviation = 0, does not add to the explain-ability of the predictions w.r.t the trained model. But it is interesting to see the weights being associated with the rejected features for my requirement.

[thomasp85]

A can sympathise with that. Unfortunately the removal is a hard requirement for the algorithm 😕

[sourabhsparkala]

No issues. I have modified the code to serve my purpose. Thank you for making the code open source and easy to understand, which helps newbies like us to learn and implement such complex tools in feasible manner.