search

yu-rp / KANbeFair

A More Fair and Comprehensive Comparison between KAN and MLP
147 stars 9 forks source link

Question about the FLOPs of KAN #11

Open Zzzsjj opened 2 months ago

Zzzsjj commented 2 months ago

Hi, the FLOPs calculation of a KAN layer is $$(d{in} \times d{out}) \times [9\times K \times (G + 1.5\times K) + 2\times G - 2.5 \times K - 1]$$. I understand the terms $$d{in} \times d{out}$$ and $$9 \times K$$ but the other parts are confusing. Could you help explain this formulation? Thanks!

yu-rp commented 2 months ago

Hi @Zzzsjj , thank you for your issue.

The calculation of the FLOPs is based on the original KAN code implementation, which includes the FLOPs for the De Boor-Cox iterative formulation, as well as the matrix multiplication of weights and the addition of biases.

The specific explanations for each component may be complex and not very intuitive, but overall, the largest proportion of FLOPs is contributed by the De Boor-Cox iterative formulation.

minhquoc0712 commented 1 month ago

Hi, can you explain why in the edge case of $K=0$, the FLOPs of B-spline function is $d{in} \times d{out} (2 G - 1)$?