search

meta-llama / llama

Inference code for Llama models
Other
56.03k stars 9.52k forks source link

torchrun #642

Open rbenami-cell opened 1 year ago

rbenami-cell commented 1 year ago

Unable to run torchrun --nnodes 1 --nproc_per_node 4 llama_finetuning.py --enable_fsdp --use_peft --peft_method lora --model_name /patht_of_model_folder/7B --pure_bf16 --output_dir Path/to/save/PEFT/model

running on Python is 3.11.4

Do I need to use a lower version of Python say 3.7

Please your assistance. Much appreciated.

Ben

tmgfo-iscte commented 1 year ago

I have the same issue.