search

princeton-nlp / LESS

[ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning
MIT License
376 stars 36 forks source link

Step 2: Building the gradient datastore: FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin' #13

Closed minchaoyue closed 7 months ago

minchaoyue commented 7 months ago

Hi， There is an error , when I run "Step 2: Building the gradient datastore" FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin' I check the folder "llama2-7b-p0.05-lora-seed3" generate from Step 1, only files optimizer.pt in checkpoint-1688.

thanks

mihara-bot commented 6 months ago

Hi, I encountered definitely the same problem as yours. Would you please kindly explain how you solved this problem? Many thanks!

minchaoyue commented 6 months ago

please refer the file "./less/scripts/train/warmup_lora_train.sh" there is an "if " if [[ $model_path == "meta-llama/Llama-2-13b-hf" ]]; then base_training_args="$base_training_args --fsdp 'full_shard auto_wrap' --fsdp_config llama2_13b_finetune"

so ,please set the path and name as "meta-llama/Llama-2-13b-hf".

mihara-bot commented 5 months ago

Thanks so much for your kind help!