Adefful
commented
1 week ago Thank you for your detailed report of the issues you're facing. Let me address the main points:
- Model loading error:
The error is likely due to an incorrect model path. In the script, we use:
parser.add_argument("--model_path", type=str, default="/home/ma-user/modelarts/inputs/model_2/")
To resolve this:
-
If running from Docker, mount the drive containing your pre-downloaded model.
-
If running locally, pass the correct path to where your model weights are stored.
- CUDA in Docker:
The "No CUDA runtime is found" message suggests CUDA might not be properly set up in your Docker environment. If you have a GPU and drivers set up on your machine, try running Docker with GPU access:
docker run --gpus all ...
This should allow the container to access your GPU.
For the PEFT library import error, please ensure you're using the latest version of the library as mentioned in the project requirements.
If you continue to face issues, please provide more details about your setup and the exact steps you're following. We'd be happy to help further.
Hi, First of all, thank you for your excellent work on this project. I tried to run your code on my machine, but I encountered an error that seems to indicate an issue with loading the model from a non-existent path. Here are the detail
Traceback (most recent call last): File "/home/anon/vul-llm-finetune/LLM/starcoder/finetune/run.py", line 418, in
main(args)
File "/home/ anon/vul-llm-finetune/LLM/starcoder/finetune/run.py", line 399, in main
run_training(args)
File "/home/ anon/vul-llm-finetune/LLM/starcoder/finetune/run.py", line 346, in run_training
model_and_data = prepare_model_and_data(args)
File "/home/ anon /vul-llm-finetune/LLM/starcoder/finetune/run.py", line 238, in prepare_model_and_data
tokenizer = AutoTokenizer.from_pretrained(args.model_path)
File "/home/ anon/miniconda3/envs/llms/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 643, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
File "/home/ anon/miniconda3/envs/llms/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 487, in get_tokenizer_config
resolved_config_file = cached_file(
File "/home/ anon/miniconda3/envs/llms/lib/python3.9/site-packages/transformers/utils/hub.py", line 417, in cached_file
resolved_file = hf_hub_download(
File "/home/ anon/miniconda3/envs/llms/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
validate_repo_id(arg_value)
File "/home/ anon/miniconda3/envs/llms/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/ma-user/modelarts/inputs/model_2/'. Use
repo_typeargument if needed.It seems the script is trying to load the model from a path that does not exist. Let me know if you can provide guidance on how to correctly specify the model path or adjust the script to handle this scenario.
Next, I attempted to run the code within a Docker container. Since this was my first time running code with CUDA in Docker, I'm unsure if the issue is related to CUDA with Docker or the PEFT library. Here is the error I encountered:
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' Traceback (most recent call last): File "/home/ma-user/modelarts/inputs/code_1/finetune/run.py", line 25, in
from peft import LoraConfig, get_peft_model, prepare_model_for_int8_training, set_peft_model_state_dict, AdaLoraConfig
ImportError: cannot import name 'prepare_model_for_int8_training' from 'peft' (/usr/local/lib/python3.8/dist-packages/peft/init.py)
This is the command used python3 /home/ma-user/modelarts/inputs/code_1/finetune/run.py --dataset_tar_gz='/home/ma-user/modelarts/inputs/data_0/java_k_1_strict_2023_07_03.tar.gz' --split="train" --seq_length 2048 --batch_size 1 --gradient_accumulation_steps 160 --learning_rate 1e-4 --lr_scheduler_type="cosine" --num_warmup_steps 1 --weight_decay 0.05 --output_dir='/home/ma-user/modelarts/outputs/results_0/' --log_freq=1 --ignore_large_functions --delete_whitespaces --base_model starcoder --lora_r 8
Let me know if you can suggest any possible solution. I am particularly concerned whether this is a compatibility issue with CUDA in Docker or a problem with the peft library itself. Thank you for your assistance!