在lcqmc数据集上微调效果下降

[elihuan1990]

在lcqmc数据集上微调simbert，在测试集上spearman指标下降一个点，怎么微调simbert呢？

[bojone]

可以用sentence-bert的方式微调

[WenTingTseng]

請問simbert.py訓練完模型並儲存best_model.weights了 我要如何加載best_model.weights模型並測試 `from bert4keras.tokenizers import Tokenizer from bert4keras.models import build_transformer_model from keras.models import Model import numpy as np

config_path = '/home/rca/research/simbert/root/kg/bert/chinese_simbert_L-12_H-768_A-12/bert_config.json' checkpoint_path = './latest_model.ckpt' dict_path = '/home/rca/research/simbert/root/kg/bert/chinese_simbert_L-12_H-768_A-12/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True)

bert = build_transformer_model( config_path, checkpoint_path, with_pool='linear', application='unilm', return_keras_model=False, ) model = Model(inputs=bert.model.inputs, outputs=bert.model.outputs) model.load_weights(checkpoint_path, by_name=True) # 加载权重时需要加上 by_name=True

test_sentence = "微信和支付宝哪个好？"

def gen_similar_sentences(text, n=10, k=10): similar_sentences = gen_synonyms(text, n, k) # 需要定义 gen_synonyms 函数 return similar_sentences

token_ids, segment_ids = tokenizer.encode(test_sentence, max_length=maxlen)

output_ids = model.predict([np.array([token_ids]), np.array([segment_ids])]) output_ids = output_ids[0].argmax(axis=1)

generated_sentence = tokenizer.decode(output_ids)

print(f"原句子：{test_sentence}") print(f"生成句子：{generated_sentence}") print("相似句子：") similar_sentences = gen_similar_sentences(test_sentence) for idx, sentence in enumerate(similar_sentences): print(f"{idx + 1}. {sentence}")` 是這樣寫嗎

[HelenGuohx]

我的方法是直接 from simbert import gen_synonyms，这样模型会加载新的权重