mzbac / llama2-fine-tune

Scripts for fine-tuning Llama2 via SFT and DPO.
189 stars 37 forks source link

After training with DPOTrainer of trl, and saving, loading error when using AutoPeftModelForCausalLM #2

Open MS-YUN opened 1 year ago

MS-YUN commented 1 year ago

After training with DPOTrainer of trl, and saving, loading error when using AutoPeftModelForCausalLM

After training with DPOTrainer of trl, I saved it locally as below and then loaded it with AutoPeftModelForCausalLM, and an error came out. When I load a checkpoint stored locally with SFTTranier in the same way, I don't get an error. I put token (no problem) and something else. but I keep getting the same error. I ask for the help. Thank you.

dpo_trainer.model.save_pretrained("./model/dpo_results/final_checkpoint")
tokenizer.save_pretrained("./model/dpo_results/final_checkpoint")

('./model/dpo_results\final_checkpoint\tokenizer_config.json', './model/dpo_results\final_checkpoint\special_tokens_map.json', './model/dpo_results\final_checkpoint\vocab.json', './model/dpo_results\final_checkpoint\merges.txt', './model/dpo_results\final_checkpoint\added_tokens.json', './model/dpo_results\final_checkpoint\tokenizer.json')

os.listdir("./model/dpo_results/final_checkpoint")

['README.md', 'adapter_model.bin', 'adapter_config.json', 'tokenizer_config.json', 'special_tokens_map.json', 'vocab.json', 'merges.txt', 'tokenizer.json']

import torch
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer, AutoModelForCausalLM

DEV = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
adapter_path = "./model/dpo_results/final_checkpoint"
model = AutoPeftModelForCausalLM.from_pretrained(
adapter_path,
torch_dtype=torch.bfloat16,
load_in_4bit=True,
token = HUGGINGFACEHUB_API_TOKEN,
)
tokenizer = AutoTokenizer.from_pretrained(adapter_path)

HTTPError Traceback (most recent call last) File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_errors.py:261, in hf_raise_for_status(response, endpoint_name) 260 try: --> 261 response.raise_for_status() 262 except HTTPError as e:

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\requests\models.py:1021, in Response.raise_for_status(self) 1020 if http_error_msg: -> 1021 raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/None/resolve/main/config.json

The above exception was the direct cause of the following exception:

RepositoryNotFoundError Traceback (most recent call last) File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\transformers\utils\hub.py:428, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs) 426 try: 427 # Load from URL or cache if already cached --> 428 resolved_file = hf_hub_download( 429 path_or_repo_id, 430 filename, 431 subfolder=None if len(subfolder) == 0 else subfolder, 432 repo_type=repo_type, 433 revision=revision, 434 cache_dir=cache_dir, 435 user_agent=user_agent, 436 force_download=force_download, 437 proxies=proxies, 438 resume_download=resume_download, 439 token=token, 440 local_files_only=local_files_only, 441 ) 442 except GatedRepoError as e:

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_validators.py:118, in validate_hf_hub_args.._inner_fn(*args, *kwargs) 116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs) --> 118 return fn(args, **kwargs)

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\file_download.py:1195, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, legacy_cache_layout) 1194 try: -> 1195 metadata = get_hf_file_metadata( 1196 url=url, 1197 token=token, 1198 proxies=proxies, 1199 timeout=etag_timeout, 1200 ) 1201 except EntryNotFoundError as http_error: 1202 # Cache the non-existence of the file and raise

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_validators.py:118, in validate_hf_hub_args.._inner_fn(*args, *kwargs) 116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs) --> 118 return fn(args, **kwargs)

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\file_download.py:1541, in get_hf_file_metadata(url, token, proxies, timeout) 1532 r = _request_wrapper( 1533 method="HEAD", 1534 url=url, (...) 1539 timeout=timeout, 1540 ) -> 1541 hf_raise_for_status(r) 1543 # Return

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_errors.py:293, in hf_raise_for_status(response, endpoint_name) 285 message = ( 286 f"{response.status_code} Client Error." 287 + "\n\n" (...) 291 " make sure you are authenticated." 292 ) --> 293 raise RepositoryNotFoundError(message, response) from e 295 elif response.status_code == 400:

RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-6508609b-66056fe260c0c09f5c752227;a56272fe-4e1b-43fc-b435-e0b47d9222f0)

Repository Not Found for url: https://huggingface.co/None/resolve/main/config.json. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated.

The above exception was the direct cause of the following exception:

OSError Traceback (most recent call last) e:\gdstrm\내 드라이브\Colab Notebooks\dl\dl_nlp\RLHF_3_DPO\2_미디엄블로그\work_미디엄블로그.ipynb Cell 84 line 7 5 DEV = torch.device('cuda' if torch.cuda.is_available() else 'cpu') 6 adapter_path = "./model/dpo_results/final_checkpoint" ----> 7 model = AutoPeftModelForCausalLM.from_pretrained( 8 adapter_path, 9 torch_dtype=torch.bfloat16, 10 load_in_4bit=True, 11 token = HUGGINGFACEHUB_API_TOKEN, 12 ) 13 tokenizer = AutoTokenizer.from_pretrained(adapter_path)

.............

OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=<your_token>

Please help me if you know how to resolve the error Thank you.

※ PS, I used "facebook/opt-350m" as base_model. and I followed this guide faithfully (https://github.com/mzbac/llama2-fine-tune)`

mzbac commented 1 year ago

The code looks correct to me. Maybe double-check your adapter_config.json to see if it has the correct base_model_name_or_path.