finetune中的继续训练问题

[Lyl-Lab]

在finetune.sh中增加以下参数，但是并没有从保存的最后一个checkpoint继续训练，请问怎么实现finetune中的继续训练呢？ --resume_from_checkpoint True \

[Lyl-Lab]

同样的，使用--output_dir previous_output_dir \ 使用这个指令也不可以实现从checkpoints恢复训练。

在代码中直接设置trainer.train(resume_from_checkpoint=True)，会报AttributeError: 'BiEncoderModel' object has no attribute '_keys_to_ignore_on_save'

[Lyl-Lab]

@staoxiao 感谢您的指导！

[staoxiao]

可以尝试增加这个值并设置_keys_to_ignore_on_save=None

[Lyl-Lab]

感谢您的回复，按照你的提示我在BiEncoderModel增加了如下代码 class BiEncoderModel(nn.Module): TRANSFORMER_CLS = AutoModel _keys_to_ignore_on_save = None
可以运行，但是有以下字段： There were missing keys in the checkpoint model loaded: ['model.embeddings.word_embeddings.weight', 'model.embeddings.position_embeddings.weight', 'model.embeddings.token_type_embeddings.weight', 'model.embeddings.LayerNorm.weight', 'model.embeddings.LayerNorm.bias', 'model.encoder.layer.0.attention.self.query.weight', 'model.encoder.layer.0.attention.self.query.bias', 'model.encoder.layer.0.attention.self.key.weight', 'model.encoder.layer.0.attention.self.key.bias', 'model.encoder.layer.0.attention.self.value.weight', 'model.encoder.layer.0.attention.self.value.bias', 'model.encoder.layer.0.attention.output.dense.weight', 'model.encoder.layer.0.attention.output.dense.bias', 'model.encoder.layer.0.attention.output.LayerNorm.weight', 'model.encoder.layer.0.attention.output.LayerNorm.bias', 'model.encoder.layer.0.intermediate.dense.weight', 'model.encoder.layer.0.intermediate.dense.bias', 'model.encoder.layer.0.output.dense.weight', 'model.encoder.layer.0.output.dense.bias', 'model.encoder.layer.0.output.LayerNorm.weight',... 请问这个是否正常？