search

princeton-nlp / LESS

[ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning
MIT License
378 stars 37 forks source link

No optimizer.bin in Step 2 #18

Open mihara-bot opened 6 months ago

mihara-bot commented 6 months ago

Hi， When I run "Step 2: Building the gradient datastore" FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin' I check the folder "llama2-7b-p0.05-lora-seed3" generate from Step 1, only files optimizer.pt in checkpoint-1688.

I noticed that in other issues, some scholars had problem on generating optimizer.pt. I think my problem is different from these. May I kindly ask for you help?

xiamengzhou commented 6 months ago

Could you confirm that you are using transformers==4.36.2? Transformers package gets frequently updated especially the part that supports distributed training :(

pengming617 commented 6 months ago

i have the same error while using transformers==4.36.2

No such file or directory: 'out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin'

xiamengzhou commented 5 months ago

@pengming617 Please refer to this thread! I think you need to use fsdp to get optimizer.pt.