how can I store model parameters ?

[imtiyazuddin]

I want to store model parameters as weights and bias files for each layer (along with function names etc), how to do it?

[KindXiaoming]

This may help: https://kindxiaoming.github.io/pykan/API_demo/API_8_checkpoint.html

[imtiyazuddin]

I want to store weights and biases as a text file , anyway to do that?

[imtiyazuddin]

basically I want to see if I can build a keras model os similar by storing the trrained KAN model weights and biases in a text file. please let me knof if there's a way to extract weights and biases as text files. thanks

[xinlnix]

basically I want to see if I can build a keras model os similar by storing the trrained KAN model weights and biases in a text file. please let me knof if there's a way to extract weights and biases as text files. thanks

Besides the weights and the biases, the calculation also important. I have the same issue to convert to c-like basical language.

[lllinlinll]

Now, I initialize a model and then load the previously trained and saved model file (using save_ckpt). However, an error occurs indicating that the dimensions do not match. How should I handle this issue?

[seyidcemkarakas]

I think here you can storage you model and try later:

here is first model and model training (I got 14 features and my case is regression case) =>

model1 = KAN(width=[14,5,1], grid=3, k=11)

results = model1.train({'train_input': train_input, 'train_label': train_label, 'test_input': val_input, 'test_label': val_label},
                      opt="LBFGS", steps=75, loss_fn=torch.nn.MSELoss()) 

after model1 training we need to save attributes that KAN model needs =>

torch.save({
    'model_state_dict': model1.state_dict(),
    'acts': model1.acts, 
    'spline_postacts':model1.spline_postacts,
    'acts_scale':model1.acts_scale
}, 'model_checkpoint.pth') # path of attributes

after saving model1 attributes we can create a new model =>

BUT main hyperparameters of KAN model needs to be defined

model2 = KAN(width=[14,5,1], grid=3, k=11)
#BUT main hyperparameters of KAN model needs to be defined

checkpoint = torch.load('model_checkpoint.pth')
model2.load_state_dict(checkpoint['model_state_dict'])
model2.acts = checkpoint['acts'] 
model2.spline_postacts = checkpoint['spline_postacts'] 
model2.acts_scale = checkpoint['acts_scale']

Now you can use model2 as it model1 (main model). You dont need to train again model2 because it is same as model1

@KindXiaoming Can you check it ? @imtiyazuddin @xinlnix @lllinlinll @hipitihop

[KindXiaoming]

@seyidcemkarakas , thank you! Are the symbolic part loaded correctly as well? (for example, model.fix_symbolic(0,0,0,'sin'), will model2's (0,0,0) be 'sin'?)

[seyidcemkarakas]

@KindXiaoming when I run this code:

model1.fix_symbolic(0,0,0,'sin')
>> r2 is 1.0000004768371582
tensor(1.0000)

and run same thing for model2 which uploaded from model1

model2.fix_symbolic(0,0,0,'sin')
>>r2 is 1.0000004768371582
tensor(1.0000)

They are exact same thing. @KindXiaoming Also Is there any thing comes to your mind that needs to be checked ? ı can check what needs to checked cuz I think most of the users want to storage their trained KAN models.

[lllinlinll]

Thank you. Additionally, I would like to ask how to determine the width of model2, especially when the network depth I defined is very large. How can I get the width of the model after completing the training? @seyidcemkarakas

[KindXiaoming]

After updating to the most-to-update version (0.2.0),

from kan.ckpt import *
path = 'model'
saveckpt(model, path)
model_loaded = loadckpt(path)