The loss doesn't decrease

[leoq276]

I tested how well KAN fits some simple functions and found that in some cases the training and testing losses do not decrease.

For example:

from kan import *
model = KAN(width=[2,5,1], grid=5, k=3, seed=0)
f = lambda x: x[:,0] + x[:,1]
dataset = create_dataset(f, n_var=2)
# train the model
result = model.train(dataset, opt="LBFGS", steps=20);
# loss
plt.plot(res['train_loss'], label='Train')
plt.plot(res['test_loss'], label='Test')
plt.legend()

This is also true for f = lambda x: torch.sqrt(x[:,0]**2 + x[:,1]**2), f = lambda x: x[:,0]**2 + x[:,1]**2 and some other cases.

I have tried to change some of the parameters ,including width, grid, opt, lr, 'lamb,lamb_l1` but it didn't work.

[KindXiaoming]

Hi, please change

f = lambda x: x[:,0] + x[:,1] to

f = lambda x: x[:,[0]] + x[:,[1]]. I apologize this could be a bit confusing.

[leoq276]

Thank you so much! It turns out using f = lambda x: x[:,0] + x[:,1] creates the label with shape [1000] and f = lambda x: x[:,[0]] + x[:,[1]] creates the label with shape [1000, 1].