search

KindXiaoming / pykan

Kolmogorov Arnold Networks
MIT License
14.75k stars 1.35k forks source link

hellokan.ipynb returns NaN instead of formula. #179

Closed Stealeristaken closed 2 months ago

Stealeristaken commented 4 months ago

Hi.

I was trying hellokan.ipynb file. I do some scaleup at training steps like 50 -> 150. In the end train_loss started to return NaN instead of any value.

I thought maybe it's a kernel error. So I re-downloaded the baseline hellokan.ipynb and rerun without any editing. It returned NaN once again. I will drop screenshot about problem.

Ekran Resmi 2024-05-13 15 08 12
KindXiaoming commented 4 months ago

Hi, the problem was caused by the appearance of the log function (which is unexpected behavior). This means that the pruning step is not good. Could you show the plot you have after pruning? From feedback from others, you may try model.prune(threshold=5e-2) instead of just model.prune().

Stealeristaken commented 4 months ago

Sorry for late response. I tried both prune options (no-specified threshold and specified threshold) result was the same i am adding pictures specified

Ekran Resmi 2024-05-16 20 46 51

no-specified

Ekran Resmi 2024-05-16 20 45 17

ShuleiCao commented 4 months ago

I encountered a similar issue, but I found that increasing the step size helped.

KindXiaoming commented 4 months ago

@Stealeristaken, in block [8], it should again specify the threshold model = model.prune(threshold=5e-2)

KindXiaoming commented 4 months ago

@ShuleiCao Thanks, yes, the pruning results can depend on quite many factors. Training longer will usually end up a sparser network.