Transfer Learning- Changing learning rate after loading the model

[sinajahangir]

Description

Is it possible to change the learning rate when a model is loaded? I was not able to find any related documentation. For example, if I train a transformer-based model on thousands of data and want to fine-tune it (retrain) for a smaller, new series, using the same training configuration as the source model (e.g., learning rate) does not make sense.

Link

No response

[marcopeix]

Hey! So, you can do this like so:

nf = NeuralForecast(models=models, freq='M')
nf.fit(df=Y_df)
nf.save()
loaded_model = nf.load()

Then, we can fit on another dataset with

loaded_model.models[0].learning_rate = 1e-4
loaded_model.fit(new_df)

I hope this helps!