search

Blealtan / efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).
MIT License
3.49k stars 306 forks source link

time complexity #7

Closed Aniosss closed 1 month ago

Aniosss commented 1 month ago

Hey, I want to use your implementation, do you know how much slower the learning can be compared to nn.linear?

akaashdash commented 1 month ago

Some benchmarks that might be helpful: https://github.com/GistNoesis/FusedFourierKAN/issues/4

Blealtan commented 1 month ago

The activation is a bit (or more, if your VRAM bandwidth is limited) slower, and the number of parameters is (by default) 5x over nn.Linear with the same input and output size, so do the FLOPs count. I didn't do real benchmarks.

Also, AFAIK chebbykan is doing the same way as my impl, which is also the second best in the benchmark by @Jerry-Master mentioned above. The results seem reasonable to me.