NTMC-Community / MatchZoo

Facilitating the design, comparison and sharing of deep text matching models.
Apache License 2.0
3.82k stars 898 forks source link

how to save the model on each epoch? #771

Closed lis-kp closed 4 years ago

lis-kp commented 4 years ago

hi, I want to save the model on each epoch. I tried:

evaluate = mz.callbacks.EvaluateAllMetrics(model, x=valid_x, y=valid_y, batch_size=16, once_every=1, model_save_path='path_to_model')

but I get the following error : TypeError: can't pickle SwigPyObject objects

Does anyone have an idea on how to solve this? thanks a lot!

bwanglzu commented 4 years ago

guess you want to use this callbacl and train your model with fit_generator(...., callbacks = [evaluate_callback, save_model_on_epoch_callback], ...)

andglf commented 4 years ago

@lis-kp 我也遇到了同样的问题,请问您找到问题所在和解决方法了吗?

uduse commented 4 years ago

Please provide more related information, preferably a code snippet that can reproduce the bug, so I can work on it.

lis-kp commented 4 years ago

I am trying to save dssm model using:

train_generator = mz.PairDataGenerator(train_processed, num_dup=1, num_neg=40, batch_size=32, shuffle=False) valid_x, valid_y = valid_processed.unpack() evaluate = mz.callbacks.EvaluateAllMetrics(model, x=valid_x, y=valid_y, batch_size=32, once_every=1,model_save_path='./model/') history = model.fit_generator(train_generator, epochs=30, callbacks=[evaluate])

the error I get is : TypeError: can't pickle SwigPyObject objects

andglf commented 4 years ago

I know where my problem is. My optimizer uses the Keras method, which is used as follows:

model.params['optimizer'] = keras.optimizers.Adam(lr=1e-3)

@lis-kp are you a similar situation?

uduse commented 4 years ago

https://github.com/NTMC-Community/MatchZoo/issues/726

On the top of your file, call matchzoo.utils.make_keras_optimizer_picklable().

Next time please submit your code together with your problem. This is a known and fixed issue. If I have known you used a customized optimizer, this could be solved in seconds.

andglf commented 4 years ago

I tried matchzoo.utils.make_keras_optimizer_picklable(),but didn't succeed. Error message:

AttributeError: module 'matchzoo.utils' has no attribute 'make_keras_optimizer_picklable'

My version is 2.1.0, is my version too low?

lis-kp commented 4 years ago

I get the same error: AttributeError: module 'matchzoo.utils' has no attribute 'make_keras_optimizer_picklable'

andglf commented 4 years ago

This is the problem of the version, the update to 2.2-dev branch will be solved. @lis-kp

lis-kp commented 4 years ago

oh, ok, thanks!