Open glotm opened 2 months ago
I saw that other people's methods include adjusting "updata_grid=False". My training after pruning was successful, but when I used the "refine" function to continue training, it was still NAN.
PS.3 I adjusted my network structure
That is, to train on a smaller-scale network. At this time, all the training is normal, and I don't need to use "updata_grid=False" anymore. This question makes me a little puzzled.
same question when I using deeper KAN
Dataset Create
I test the effect of KAN through artificial data, and the process guarantees that NAN will not appear.
Training
when I first train,this is look fine
Problem
Then I use the "prune" function,and train again,the loss start begin NAN I adjusted the learning rate of all the fit functions above to 0.001, but the training after pruning is still nan.
How can I solve this problem