search

KindXiaoming / pykan

Kolmogorov Arnold Networks
MIT License
13.88k stars 1.25k forks source link

What does `reg` mean when training? #221

Closed egmaminta closed 1 week ago

egmaminta commented 2 months ago

Hi! I would like to ask what does reg mean when training KAN? For example you would see something like:

train loss: xxx | test loss: yyy | reg: zzz : ...

And why is it always changing? Does this refer to the lambda or overall regularization parameter?

KindXiaoming commented 2 months ago

sorry for the confusion. reg means the regularization term after the lambda coefficient. Regularization could include l1, entropy, etc.

egmaminta commented 2 months ago

Oh I see... if reg is the regularization term, shouldn't it be equal to zero if I set the lambda coefficient (the coefficient that refers to the overall regularization magnitude) equal to zero as well?

KindXiaoming commented 2 months ago

oh sorry, so the displayed one is before applying the coefficient lamb, i.e., real_reg = lamb * reg, total_loss = pred_loss + real_reg