ERROR at the saving stage in the starcoder finetune scripts

bigcode-project / octopack

🐙 OctoPack: Instruction Tuning Code Large Language Models

https://arxiv.org/abs/2308.07124

MIT License

420 stars 27 forks source link

ERROR at the saving stage in the starcoder finetune scripts #20

Open ttssp opened 1 year ago

ttssp commented 1 year ago

Hi friends,

I was trying the finetuing/starcoder/finetune.py using torch.distributed.launch, and met the following error : FileNotFoundError: [Errno 2] No such file or directory: './checkpoints/checkpoint-10/pytorch_model.bin' caused at /finetuning/starcoder/finetune.py", line 35.

Could anyone let me know how to solve it?

ttssp commented 1 year ago

not quite sure what is pytorch_model.bin, when it is generated and why we should remove it..

ArmelRandy commented 1 year ago

Hi. This part of the code is due to the fact that we wrote it when the peft library was still nascent. With recent version of transformers and peft, you don’t have to create your own callbacks. Feel free to remove this callback from your trainer.

ttssp commented 1 year ago

Thanks a lot for the reply. I'll try right now.