search

Ekeany / Boruta-Shap

A Tree based feature selection tool which combines both the Boruta feature selection algorithm with shapley values.
MIT License
559 stars 86 forks source link

Enable logistic regression as supported model #39

Open yedide opened 3 years ago

yedide commented 3 years ago

Hi,

Would it be possible to add the option of using Logistic regression models in Boruta Shap?

Thanks!

Ekeany commented 3 years ago

Hi,

It is definitely possible to do, we just need to replace the feature importance metric with the coefficients themselves or use SHAP's linear explainer to calculate some measure of feature importance.

However, Linear models are much more transparent than ensemble based models and do not consider interactions between features. I would normally just use an L1 or L2 penalty to regularize the model and then use the information about the coefficients to select an appropriate subset or leave as is.

What are your thoughts ?

yedide commented 3 years ago

Hi, Yes this is true that LR models are more transparent than RF or GBoost. But e.g I would like to publish a paper for mortality prediction and I try multiple models to predict. The feature selection step is based on each model separately. That's why I would have to apply a common approach for all the models and Boruta-Shap is potentially applicable for all models. Basically would it be possible to use Boruta-Shap for Naive Bayes models ? Thanks !

Ekeany commented 3 years ago

Hi Yeddie,

It is possible to use SHAP with any model however it will not be optimized and will be incredibly slow !.

How large is your feature set? Could you consider doing some sort of recursive elimination ?

Eoghan.

On Thu, Jan 28, 2021 at 1:29 PM yedide notifications@github.com wrote:

Hi, Yes this is true that LR models are more transparent than RF or GBoost. But e.g I would like to publish a paper for mortality prediction and I try multiple models to predict. The feature selection step is based on each model separately. That's why I would have to apply a common approach for all the models and Boruta-Shap is potentially applicable for all models. Basically would it be possible to use Boruta-Shap for Naive Baies models ? Thanks !

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Ekeany/Boruta-Shap/issues/39#issuecomment-769053023, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMDEERRJLQ365AQGDZ6SHOTS4FRE7ANCNFSM4WTV3NYA .