Closed minchaoyue closed 3 months ago
Hi, I encountered definitely the same problem as yours. Would you please kindly explain how you solved this problem? Many thanks!
please refer the file "./less/scripts/train/warmup_lora_train.sh" there is an "if " if [[ $model_path == "meta-llama/Llama-2-13b-hf" ]]; then base_training_args="$base_training_args --fsdp 'full_shard auto_wrap' --fsdp_config llama2_13b_finetune"
so ,please set the path and name as "meta-llama/Llama-2-13b-hf".
Thanks so much for your kind help!
Hi, There is an error , when I run "Step 2: Building the gradient datastore" FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin' I check the folder "llama2-7b-p0.05-lora-seed3" generate from Step 1, only files optimizer.pt in checkpoint-1688.
thanks