search

KindXiaoming / pykan

Kolmogorov Arnold Networks
MIT License
13.92k stars 1.25k forks source link

loss_fn by default is Mean Squared Error (MSE), not Root Mean Squared Error (RMSE)? #32

Closed mw66 closed 1 week ago

mw66 commented 2 months ago

https://github.com/KindXiaoming/pykan/blob/915726a6770bf439021f1bac38bfe39fd424159e/kan/KAN.py#L844

Also can you add the doc for this loss_fn param?

KindXiaoming commented 2 months ago

Hi the training loss is by default MSE, but the saved results are for RMSE. RMSE loss is not differentiable (at zero) hence would have optimization problems. MSE has better landscape.

mw66 commented 2 months ago

Hi the training loss is by default MSE, but the saved results are for RMSE. RMSE loss is not differentiable (at zero) hence would have optimization problems. MSE has better landscape.

OK, but can you doc this info, otherwise it's confusing when comparing the above code with the paper / tutorials.