search

meta-llama / llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.
9.95k stars 1.4k forks source link

Success in one command, failed using other command #518

Open Tizzzzy opened 2 weeks ago

Tizzzzy commented 2 weeks ago

System Info

  1. python: 3.10.12
  2. nvcc: 
    nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0
  3. peft: 0.10.0

All other packages are the same as the requirments.txt

My local machine has 32G RAM. My gpu information: NVIDIA-SMI 550.54.15; Driver Version: 545.84; CUDA Version: 12.3; NVIDIA GeForce RTX 3070; 8192MiB

Information

🐛 Describe the bug

I am new to llama_recipes, and I am trying to finetune llama3 on a new dataset. Following the finetune tutorial, I use the command python -m llama_recipes.finetuning --use_peft --peft_method lora --quantization --model_name meta-llama/Meta-Llama-3-8B --dataset "openbookqa_dataset" --custom_dataset.file "datasets/openbookqa_dataset.py" --batching_strategy "packing" to finetune llama3. This command successfully compiled. But the problem is, every time I make a change to the code on my local machine, I need to call this command first git checkout main && git pull && pip install -U ..

Then I see in singlegpu_finetuning page, there is a new command: python -m finetuning.py --use_peft --peft_method lora --quantization --model_name meta-llama/Meta-Llama-3-8B --dataset "openbookqa_dataset" --custom_dataset.file "datasets/openbookqa_dataset.py" --batching_strategy "packing". I am assuming this command will run my local finetuning python file in this path ~\llama-recipes\src\llama_recipes\finetuning.py. However, this command gives me error.

Error logs

(llama3) root@Dong:/mnt/c/Users/super/OneDrive/Desktop/research/llama-recipes/src/llama_recipes# python -m finetuning.py --use_peft --peft_method lora --quantization --model_name meta-llama/Meta-Llama-3-8B --dataset "openbookqa_dataset" --custom_dataset.file "datasets/openbookqa_dataset.py" --batching_strategy "packing"
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/mnt/c/Users/super/OneDrive/Desktop/research/llama-recipes/src/llama_recipes/finetuning.py", line 31, in <module>
    from llama_recipes.utils import fsdp_auto_wrap_policy
  File "/usr/local/lib/python3.10/dist-packages/llama_recipes/utils/__init__.py", line 5, in <module>
    from llama_recipes.utils.dataset_utils import *
  File "/usr/local/lib/python3.10/dist-packages/llama_recipes/utils/dataset_utils.py", line 10, in <module>
    from llama_recipes.datasets import (
  File "/usr/local/lib/python3.10/dist-packages/llama_recipes/datasets/__init__.py", line 4, in <module>
    from llama_recipes.datasets.grammar_dataset.grammar_dataset import get_dataset as get_grammar_dataset
  File "/usr/local/lib/python3.10/dist-packages/llama_recipes/datasets/grammar_dataset/grammar_dataset.py", line 8, in <module>
    from datasets import load_dataset
  File "/mnt/c/Users/super/OneDrive/Desktop/research/llama-recipes/src/llama_recipes/datasets/__init__.py", line 4, in <module>
    from llama_recipes.datasets.grammar_dataset.grammar_dataset import get_dataset as get_grammar_dataset
ImportError: cannot import name 'get_dataset' from partially initialized module 'llama_recipes.datasets.grammar_dataset.grammar_dataset' (most likely due to a circular import) (/usr/local/lib/python3.10/dist-packages/llama_recipes/datasets/grammar_dataset/grammar_dataset.py)

Expected behavior

I expect both commands will work

wukaixingxp commented 2 weeks ago

Hi! python -m will run the installed module and you have to pip install to update llama-recipe module. Alternatively you can just run the python file. In the main llama_recipes folder, please run python recipes/finetuning/finetuning.py --use_peft --peft_method lora --quantization --model_name meta-llama/Meta-Llama-3-8B --dataset "openbookqa_dataset" --custom_dataset.file "src/llama_recipes/datasets/openbookqa_dataset.py" --batching_strategy "packing" instead of running the src/llama_recipes/finetuning.py. Let me know if there is any problem.