search

csinva / imodels

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).
https://csinva.io/imodels
MIT License
1.34k stars 119 forks source link

Is it worth implementing or importing KAN? #208

Closed helq closed 2 weeks ago

helq commented 4 weeks ago

The original implementation of KANs (Kolmogorov-Arnold Network) claims to generate interpretable and as good as MLP models. A KAN could be several layers deep and perform as well as a neural network. This is quite an awesome claim. It means having more complex models that could be also "interpretable".

If I am correct, what the original implementation does (KindXiaoming/pykan) is to train a KAN, then figure out which connections are the least relevant and chop them off. More retraining and some symbolic function fitting later results in a closed formula!

I am wondering what your opinion on this approach is and whether it would make any sense to implement KANs in imodels.

Thank you for your awesome package!!

csinva commented 4 weeks ago

Hello, thanks for your interest! We actually implemented KAN in our sister package imodelsX (see demo notebook here). We are keeping things with heavy dependencies (like pytorch) in that repo rather here in imodels.

helq commented 4 weeks ago

I see. Cool. I find the "about" paragraph in imodelsX to be a bit confusing. It talks about LLMs but KANs are implemented in there. You could break the table presenting all the models into two: explainable modeling and explainable steering, as to make clear in which camp each one of those fall. Thanks for the quick response :)