Missing dictionary: VOCAB，MERGES and other files during fine-tuning

[qiu6669]

Hello, may I ask where the VOCAB and Merge files are located in the second step of fine-tuning the dictionary in this task using a specific biological dataset? The model trained in the first step does not have this dictionary, so I tried using the pre trained dictionary from the first step and found that it still reported an error saying that the dictionary is damaged or not matching. The loading path is correct, but after the first step is completed, there is no dictionary file generated. I hope the blogger can give me a big answer. Thank you!!

[YinYuBB]

Thanks for your interest in this project. The vocab.json and merge.txt files used during the fine-tuning phase are the same as those in the pre-trained model RoBERTa-large-PM-M3-Voc-hf. You can save the model after the first phase, copy these two dictionary files, and then fine-tune the saved model.

To achieve this, you can follow these steps: multi-task learning. Use the save_steps argument to specify when the model should be saved, and then, in the fine-tuning phase, provide the path to the saved model. Additionally, please ensure that you copy the vocab.json and merge.txt files into the directory of the saved model from the first phase.

[qiu6669]

谢谢您的答复，我之前尝试用过您说的方法将roberta的字典复制到训练好的模型文件夹进行微调，可是它报错说字典和模型不匹配，我后面再去试试吧

[YinYuBB]

Feel free to try our model training process and reach out if you have any further questions or if the issue persists. Thanks!

[qiu6669]

您好，我按照您说的将两个预训练文件加到训练好的模型文件后微调仍然报错，具体如下： Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): line 2225, in from_pretrained model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model( ValueError: The state dictionary of the model you are trying to load is corrupted. Are you sure it was properly saved? Traceback (most recent call last): line 2225, in from_pretrained

Message ID: @.***>

[YinYuBB]

Hi, I've uploaded a detailed training demo. This demo successfully runs our two-stage model training procedures. Please refer to the demo and feel free to give it a try. If the issue persists, please provide not only the error details but also the command you ran, the saved model directory, and your environment information. Thanks!