search

SynodicMonth / ChebyKAN

Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.
350 stars 36 forks source link

Use properly tuned baselines! #1

Closed usamec closed 6 months ago

usamec commented 6 months ago

Your MLP baseline in functional interpolation is big and undertrained.

This is properly tuned MLP: https://colab.research.google.com/drive/1wJFhSeTF9xTikN_ranR2xebf9HEaHo5Y

SynodicMonth commented 6 months ago

My fault. I was initially using [1, 125, 256, 1] and even that was too big. But KAN do converge faster than MLP. (or maybe its lr problem) Ill take some test later and fix it soon.

iiisak commented 6 months ago

Even with a properly tuned MLP, ChebyKAN seems to converge faster. https://colab.research.google.com/drive/1dco5uoDXSF7c6B2WDGhabT7dnsAJiOzi