search

interpretml / interpret

Fit interpretable models. Explain blackbox machine learning.
https://interpret.ml/docs
MIT License
6.04k stars 714 forks source link

Incremental fitting of interactions #488

Open sadsquirrel369 opened 6 months ago

sadsquirrel369 commented 6 months ago

Is it possible to first fit a model without interactions and after that add interactions as a separate step?

paulbkoch commented 6 months ago

This is not yet available. There are discussions of this in https://github.com/interpretml/interpret/issues/304 and https://github.com/interpretml/interpret/issues/403

Today if you want to pick your own interactions, you need to train the model a second time specifying the interactions explicitly on the second run. It works in most scenarios, with the downside of taking more time.

paulbkoch commented 5 months ago

To be more exact, it is possible to train an initial EBM and then boost another EBM model on top of it using the init_score parameter of the fit function. You can even merge the two models afterwards using the merge_ebms function, but we would like to make this process simpler by implementing warm_start, which is how other scikit-learn models handle this scenario. You also currently lose a bit of information in the merge, like the histograms.

sadsquirrel369 commented 5 months ago

So in theory this can be done, if we remove all the features of the 'incremental' model post fit (or rather keep 1 bin per variable and overwrite the histgrams to be near zero) and then perform a merge of the ebms. But I suppose then you have the problem of not having interactions in the base model and the merge procedure will fail.

Is there an easy way of just fitting interactions?

paulbkoch commented 5 months ago

I made an example notebook to illustrate how you'd build custom pairs yourself and boost the pairs on top of the mains model. Just for fun I added some triples as well since I thought it would be interesting to many of our users.

The notebook is here: https://interpret.ml/docs/python/examples/custom-interactions.html

DerWeh commented 4 months ago

@paulbkoch I'm interested in warm-starting the EBM. If this is possible using the init_score parameter or the merge_ebms function, this would be good-enough for me. Sadly, the link to your example notebook seems not to be working anymore. Could you please update the link (maybe also giving the link to a specific commit to have a permanent link)?

paulbkoch commented 4 months ago

Thanks @DerWeh -- I've updated the link above to point to our documentation, which now has this example. At the top of the documentation link you can find a link to the example notebooks folder in our repo if you'd prefer to start from there.