search

BrikerMan / Kashgari

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.
http://kashgari.readthedocs.io/
Apache License 2.0
2.38k stars 439 forks source link

model.save：TypeError: Object of type module is not JSON serializable #461

Open asd8095075 opened 3 years ago

asd8095075 commented 3 years ago

from kashgari.corpus import ChineseDailyNerCorpus from kashgari.embeddings import BertEmbedding from kashgari.tasks.labeling import BiLSTM_CRF_Modelb import kashgari

X_train, y_train = ChineseDailyNerCorpus.load_data('train') x_test, y_test = ChineseDailyNerCorpus.load_data('test') X_valid, y_valid = ChineseDailyNerCorpus.load_data('valid')

bert_embed = BertEmbedding('./bert/chinese_roberta_wwm_ext_L-12_H-768_A-12', task=kashgari.tasks.labeling, sequence_length=100)

model = BiLSTM_CRF_Model(bert_embed) model.fit(X_train, y_train, X_valid, y_valid, epochs=1,batch_size=512)

asd8095075 commented 3 years ago

Version being used： kashgari2.0.0 Operating system： windows 10 Version of Python： 3.7.2 Errors in console： TypeError: Object of type module is not JSON serializable Detailed description of the problem： use chinese_roberta_wwm_ext_L-12_H-768_A-12，save model fail

Cliccker commented 3 years ago

I've met the same problem while training a labeling model. Did you find any solution?

asd8095075 commented 3 years ago

The problem remains unsolved

cccccrj commented 2 years ago

also encountered this problem.

HuizhiXu commented 2 years ago

got this problem too

HuizhiXu commented 2 years ago

I tried to find why this happened.

I have located the problem, it’s in this file “lib/python3.6/site-packages/kashgari/tasks/abs_task_model.py” in function save , line 84, f.write(json.dumps(self.to_dict(), indent=2, ensure_ascii=False)).

The structure of “module” might have wrong type, not JSON, so it cannot be serializied.

Also noticed that Keras has the same issue. https://github.com/keras-team/keras/issues/9342 I’ve changed the source file of Keras, but nothing has changed.

my Setup： tf 2.2.0 Ubuntu 20.04 kashgari 2.0.2
keras 2.3.1 cuda toolkit 10.1 cuDNN 7.6

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

cjrcjr commented 4 months ago

我把 bert_embed = BertEmbedding('./bert/chinese_roberta_wwm_ext_L-12_H-768_A-12', task=kashgari.tasks.labeling, sequence_length=100) 改为 bert_embed = BertEmbedding('./bert/chinese_roberta_wwm_ext_L-12_H-768_A-12') 就能成功保存模型了，反正不传入task就可以保存