search

brightmart / albert_zh

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型
https://arxiv.org/pdf/1909.11942.pdf
3.94k stars 753 forks source link

文本分类遇到ValueError: Layer weight shape (21128, 312) not compatible with provided weight shape (21128, 128) #113

Open wilsonlsm006 opened 4 years ago

wilsonlsm006 commented 4 years ago

之前使用keras_bert做文本分类任务，使用的是原始的bert预训练模型。现在想用albert，出现这个问题，代码没有改动，只改了预训练的模型： Traceback (most recent call last): File "keras_train_classification.py", line 259, in model_train(TRAIN_DATA_PATH, MODEL_SAVE_PATH) File "keras_train_classification.py", line 241, in model_train run_cv(n_fold, DATA_LIST_train, model_save_path_finame, epoch_num) File "keras_train_classification.py", line 149, in run_cv model = build_bert(NCLASS) File "keras_train_classification.py", line 193, in build_bert bert_model = load_trained_model_from_checkpoint(config_path, checkpoint_path, seq_len=None) File "/opt/conda/lib/python3.7/site-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training) File "/opt/conda/lib/python3.7/site-packages/keras_bert/loader.py", line 80, in load_model_weights_from_checkpoint loader('bert/embeddings/word_embeddings'), File "/opt/conda/lib/python3.7/site-packages/keras/engine/base_layer.py", line 1126, in set_weights 'provided weight shape ' + str(w.shape)) ValueError: Layer weight shape (21128, 312) not compatible with provided weight shape (21128, 128) oppoer@falcon:zhaoshangjiameng_albert$

brightmart commented 4 years ago

使用的配置文件也要跟上，详见【下游任务】示例

ghost commented 3 years ago

我也遇到了上面一样的问题，keras_bert支持albert预训练模型吗？如果支持的话，你说的那种解法（使用的配置文件也要跟上，详见【下游任务】示例），可以说的更清楚一点吗？下游任务示例里面没有找到解决方案