search

juntang-zhuang / Adabelief-Optimizer

Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"
BSD 2-Clause "Simplified" License
1.05k stars 108 forks source link

Tensorflow restoration issue #59

Closed soumen-ghosh closed 2 years ago

soumen-ghosh commented 2 years ago

Hi,

I've installed "adabelief-tf==0.2.0" through conda. And the tensorflow version I'm using is 2.8.0 (tensorflow==2.8.0). I'm facing below restoration issue while trying to try the model on test dataset - the checkpoint was created during training -

ValueError: Unknown optimizer: AdaBeliefOptimizer. Please ensure this object is passed to thecustom_objectsargument.

I'm using it below way:

lr_decayed_fn = tf.keras.optimizers.schedules.CosineDecayRestarts(initial_learning_rate=1e-3, first_decay_steps=200)

model.compile(loss='binary_crossentropy', 
              optimizer=AdaBeliefOptimizer(learning_rate=lr_decayed_fn, epsilon=1e-14, rectify=False),
              metrics=['accuracy', tf.keras.metrics.AUC(name="AUC")],
              )

I want to explore CosineDecayRestarts and AdaBeliefOptimizer together. Is there something wrong with my usage?

soumen-ghosh commented 2 years ago

After using below code in the test function, the restoration worked - running prediction on test dataset:

model = tf.keras.models.load_model(modelFileName, custom_objects={'AdaBeliefOptimizer': AdaBeliefOptimizer})