Add KAN (Kolmogorov-Arnold Network) to Darts

[PapadopoulosDimitrios]

There has been a recent boom of positive comments on the KAN (Kolmogorov-Arnold Network) model. Are you thinking about incorporating it? Repo: https://github.com/KindXiaoming/pykan

[dennisbader]

Hi @PapadopoulosDimitrios, this could indeed be a great addition to Darts. I'll put it in our backlog.

However, I'm not sure about adding pykan as a dependency, since their dependencies are very strict.

I think we need to investigate a bit more. Maybe we could re-implement only the relevant parts for Darts 🤔

Is this something you'd be willing to contribute to?

[tRosenflanz]

Tried replacing nn.Linear with KANLinear from here https://github.com/Blealtan/efficient-kan in the TSMixer and it works quite well. The speed of iterations is slower (clock wise) as expected but fits much quicker per iteration (metrics wise) and I get a much higher overfit suggesting that they are considerably more expressive. It didn't improve my results due to overfitting but I believe it will vary from dataset to dataset

Note: had to fix this https://github.com/Blealtan/efficient-kan/issues/36 to make it work but thats a one line change.

[PapadopoulosDimitrios]

@dennisbader Unfortunately, I won't be able to contribute due to time constraints.

FYI there's also an sklearn-style implementation ok KAN in: https://github.com/csinva/imodelsX/blob/master/imodelsx/kan/kan_sklearn.py

The general RegressionModel wrapper could be used here.

[joshua-xia]

There is TKAN implementation base on tensorflow: code: https://github.com/remigenet/TKAN and paper: https://arxiv.org/pdf/2405.07344

[joshua-xia]

another paper: https://arxiv.org/html/2405.08790v1