[Question]: MoodelCheckpoint() alternatives

[Tvvgge]

Description

Hello, I would like to save the weights for each epoch as the code I used in TensorFlow shown below. However, it appears that TensorFlow.NET does not currently support the ModelCheckpoint() function. Therefore, I would like to know if there are any alternatives or other methods to achieve the same operation.

checkpoint_path = "training/cp-{epoch:04d}.ckpt" cp_callback = tf.keras.callbacks.ModelCheckpoint( filepath=checkpoint_path, verbose=0, save_weights_only=True, save_freq='epoch') model.save_weights(checkpoint_path.format(epoch=0))

Thank you

Alternatives

No response

[Wanglongzhi2001]

Hello, the alternative method is to use model.save_weights to save model weights in Keras format(.h5 format), but due to historical reasons, TensorFlow.NET's model. load_weights exists some issues when load the weights, so I would recommend using model.save and keras.models.load_model to save and load the entire model, although this will use more memory space.

[Tvvgge]

Hello longzhi, thank you for the information! It seems that the standard methods model.save_weights and model.save are only capable of saving the model once the training is completed. However, I'm interested in finding a way to save the model after each epoch. After some research, I came across the EarlyStopping callback in src/TensorFlowNET.Keras/Callbacks/Earlystopping.cs which might hold the solution I'm looking for. However, I'm unsure about how to modify the code to achieve the desired outcome. I have tried something like:

CallbackParams callback_parameters = new CallbackParams 
 { 
     Model = model, 
     Epochs = num_epochs,
     model.save_weights($"weights{epoch:02d}.h5"); 
 };
ICallback earlystop = new EarlyStopping(callback_parameters, "loss"); 
var callbacks = new List<ICallback>{ earlystop};

but it just doesn't work for me.

Thank you

[Wanglongzhi2001]

Sorry for didn't take a closer look at your question. The CallbackParams class can't receive model.save_weights($"weights{epoch:02d}.h5");. You can refer it's definition: https://github.com/SciSharp/TensorFlow.NET/blob/adc90af2ddbeb63f4dd0a3199c8cfa469af9d7af/src/TensorFlowNET.Keras/Callbacks/CallbackParams.cs#L8-L14

Because EarlyStopping has already been encapsulated and looks like its functionality does not meet your requirements --- saving weights in each epoch. So you can define custom callbacks. Most of your custom callback's code will be the same as Earlystopping. Just copy the Ealystopping's code and modify the constructor, then invoke model.save_weights in on_epoch_begin() method and remove unnecessary code. For more information, please refer to the TensorFlow official document .

[Tvvgge]

Hello longzhi, problem solved! thank you so much for the detailed explanation!!!