search

NVlabs / DoRA

[ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation
https://nbasyl.github.io/DoRA-project-page/
Other
588 stars 36 forks source link

RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())] #6

Closed 2tianyao1 closed 4 months ago

2tianyao1 commented 4 months ago

Dear author, thank you for your outstanding contribution! We now want to reproduce the results of fine-tuning llama3-8B with dora.

We first downloaded the pre-trained weights of llama3-8B from huggingface and tried the initial weights provided by hf and original meta. Replace the base_model parameter with the downloaded weight path. But we always get this error.

The error shows that loading tokenizer failed. This seems to be caused by a failure to parse the model file tokenizer.model file. We sincerely look forward to your reply!

image

nbasyl commented 4 months ago

@2tianyao1 Hi, thanks for raising this issue, I just realize that I forgot to update the commonsense_evaluate.py when I last added support for llama3. I have just updated the file, could you try again and let me know if you encounter any issues.

2tianyao1 commented 4 months ago

Thank you very much for your reply！！ I've resolved my issue. Wishing you the very best. ❤🎊✨