ChebyKAN is faster

[Jerry-Master]

Right now the ChebyKAN implementation is the fastest one. You are not including it in your README. If you want to claim that you are the fastest you should provide evidence that supports it. Specially now that I have run the benchmark on my machine and ChebyKAN appears to run faster than yours. The updated benchmarks in my repo is now as follows:

                     |      forward  |     backward  |      forward  |     backward  |   num params  |  num trainable params
----------------------------------------------------------------------------------------------------------------------------------
effkan-cpu           |     31.98 ms  |     44.49 ms  |       nan GB  |       nan GB  |     10010000  |              10010000
effkan-gpu           |      4.76 ms  |      4.54 ms  |      0.13 GB  |      0.19 GB  |     10010000  |              10010000
fourierkan-cpu       |    727.35 ms  |    936.78 ms  |       nan GB  |       nan GB  |     10011001  |              10011001
fourierkan-gpu       |     17.93 ms  |     14.40 ms  |      1.96 GB  |      2.01 GB  |     10011001  |              10011001
fusedfourierkan-cpu  |    908.43 ms  |   1637.14 ms  |       nan GB  |       nan GB  |     10011001  |              10011001
fusedfourierkan-gpu  |     30.30 ms  |     84.61 ms  |      0.09 GB  |      0.13 GB  |     10011001  |              10011001
cufkan-cpu           |   1467.37 ms  |   3767.40 ms  |       nan GB  |       nan GB  |     10011001  |              10011001
cufkan-gpu           |      5.95 ms  |     49.74 ms  |      0.09 GB  |      0.13 GB  |     10011001  |              10011001
chebykan-cpu         |     20.29 ms  |     12.38 ms  |       nan GB  |       nan GB  |     10010000  |              10010000
chebykan-gpu         |      1.03 ms  |      1.21 ms  |      0.14 GB  |      0.13 GB  |     10010000  |              10010000
fast-kan-cpu         |      9.96 ms  |     17.06 ms  |       nan GB  |       nan GB  |     10015019  |              10015001
fast-kan-gpu         |      1.44 ms  |      2.13 ms  |      0.11 GB  |      0.14 GB  |     10015019  |              10015001
faster-kan-cpu       |     10.58 ms  |     15.42 ms  |       nan GB  |       nan GB  |     10014022  |              10014000
faster-kan-gpu       |      1.20 ms  |      2.01 ms  |      0.12 GB  |      0.14 GB  |     10014022  |              10014000
rbf-kan-cpu          |     12.59 ms  |     12.07 ms  |       nan GB  |       nan GB  |     10011019  |              10011001
rbf-kan-gpu          |      1.12 ms  |      2.08 ms  |      0.11 GB  |      0.13 GB  |     10011019  |              10011001
----------------------------------------------------------------------------------------------------------------------------------
mlp-cpu              |      9.77 ms  |      7.27 ms  |       nan GB  |       nan GB  |     10020001  |              10020001
mlp-gpu              |      0.49 ms  |      1.07 ms  |      0.10 GB  |      0.13 GB  |     10020001  |              10020001
----------------------------------------------------------------------------------------------------------------------------------
pykan-cpu            |     15.59 ms  |     17.53 ms  |       nan GB  |       nan GB  |         2431  |                  1551
pykan-gpu            |     50.56 ms  |     93.93 ms  |      0.02 GB  |      0.02 GB  |         2431  |                  1551

As you can see, the rbf-kan is also faster in the forward while being slower in the backward. And the chebykan is faster in both cases.

[AthanasiosDelis]

That's great news. I have not updated the repo since 05/17/2024, so I have not kept up with the updates. Because of the speed with which everything unfolds, I included on purpose the date of the last benchmark, which is 5/17/2024. So, I think your observation is as of May 20 or 21, correct? Unfortunately, I will be working on this project again in the first week of June due to work obligations. Keep in mind I write this comment on the road, just to be clear that there is no malintention, just a workload-no update issue. I will update my benchmarks and ranking when I catch up with the project again. I have a new version locally and have not tested for speed. Until then, I think this issue clarifies the point. I am happy that you do benchmarks again, because I think you too did not update your repo for some days, @Jerry-Master in the previous week, if I recall correctly. (rbf-kan)[https://github.com/sidhu2690/RBF-KAN/tree/main] I think it is a code restructuring of fast-kan without citation to the original; there was a related issue in pykan. Also, I do not know if you have caught up, but we have a new paper in KANs, Wav-KAN: Wavelet Kolmogorov-Arnold Networks. If there is a github repo, it would be cool to know its speed, if you have time to check before me.

[Jerry-Master]

I think you too did not update your repo for some days

I update my repo based on suggestions. I am not actively researching this area, when somebody creates an issue in my repo I try to include it. But I won't be looking around to see what has changed. I am like a process in STOPPED state waiting for a SIGCONT signal.

we have a new paper in KANs, Wav-KAN: Wavelet Kolmogorov-Arnold Networks

It seems like there is a repo here for that. I may add it in the future. Personally I think that paper should not pass any peer review process. Writing a paper in under two months is not a good idea. The production quality of that article is very low. And the experimental section is the bare minimum. Research should be done with patience. The urge to publish just degrades the quality of the results. People seem so afraid of missing out that they miss out good results mainly due to that fear alone. Instead of publishing the first thing that comes to your mind I believe it is better to mature the idea.

if I recall correctly. (rbf-kan)[https://github.com/sidhu2690/RBF-KAN/tree/main] I think it is a code restructuring of fast-kan without citation to the original; there was a related issue in pykan.

Like I mentioned above, people is in such a rush that they miss on good practices too. I hope my repo helps in giving proper atribution to every author.

[Jerry-Master]

I added Wav-Kan to the benchmark. Very slow at the moment.