search

Blealtan / efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).
MIT License
3.49k stars 306 forks source link

KAN intuition for setting the hidden layers #14

Closed marcopeix closed 1 month ago

marcopeix commented 1 month ago

Hello,

Is there a rule of thumb or intuition for setting the layers_hidden parameter? I'm using it for time series, and I use [input_size, 10, horizon]. The 10 is arbitrary, and taken from the MNIST example, but do you have a suggestion on setting these for best performance?

hoangthangta commented 1 month ago

So far, I know that the number of hidden neurons can affect performance. In the MNIST example, I tried with the size 128, 256, and 512 instead of 64. However, there is a limit. Not sure with other parameters.

model = KAN([28 * 28, 256, 10])