fyi - relu_kan claims to improve performance over bsplines

[johndpope]

https://github.com/quiqi/relu_kan/tree/main

@yu-rp claims that MLP performs better - can you test against relu_kan? https://github.com/yu-rp/KANbeFair

noticed that there's also sine_kan.

@ereinha - i saw your paper - does it underperform against mlp ? https://github.com/ereinha/SineKAN

@Jerry-Master - time for re-benchmarking? https://github.com/Jerry-Master/KAN-benchmarking

[Jerry-Master]

The repositories you mention do not seem to innovate much. In fact, the relu one is not even coded properly, it has an issue with multiple dimensions: https://github.com/quiqi/relu_kan/issues/1 Nevertheless I have included them in my benchmarks in case you are curious. Sine Kan appears to be slightly faster, but the research does not look polished. More work is needed to see if it avoids catastrophic forgetting, if it is valid for symbolic learning and if it is suitable for pruning. I think that all the functions that could be used have already been used, and effort should be put not on speed but on the other types of performance. The speed is already on par with MLP.

[johndpope]

new paper - UKAN - KAN for GPU

....made by (checks notes) NVIDIA

https://arxiv.org/pdf/2408.11200

I wonder if we are we going to see KANs baked into a future release of Pytorch..... take your bets....