finetuned model inference error

[sea-hot]

I am following the inference in the bt_generator folder. I proceeded sequentially according to the sections: install - CodeLlama Prompt - one shot - Base model inference. However, an error occurred in the finetuned model inference. Is there a way to solve this?" "I am using ROS 2 Humble on WSL2 Ubuntu 22.04." "I deeply appreciate your help."

{ "name": "ValueError", "message": "Can't find 'adapter_config.json' at 'lora_adapters/codellama-bt-adapter'", "stack": "--------------------------------------------------------------------------- HTTPError Traceback (most recent call last) File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py:304, in hf_raise_for_status(response, endpoint_name) 303 try: --> 304 response.raise_for_status() 305 except HTTPError as e:

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/requests/models.py:1021, in Response.raise_for_status(self) 1020 if http_error_msg: -> 1021 raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/lora_adapters/codellama-bt-adapter/resolve/main/adapter_config.json

The above exception was the direct cause of the following exception:

RepositoryNotFoundError Traceback (most recent call last) File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/peft/config.py:197, in PeftConfigMixin._get_peft_type(cls, model_id, hf_hub_download_kwargs) 196 try: --> 197 config_file = hf_hub_download( 198 model_id, 199 CONFIG_NAME, 200 hf_hub_download_kwargs, 201 ) 202 except Exception:

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:119, in validate_hf_hub_args.._inner_fn(*args, *kwargs) 117 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs) --> 119 return fn(args, **kwargs)

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/file_download.py:1403, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, headers, legacy_cache_layout, endpoint) 1401 elif isinstance(head_call_error, RepositoryNotFoundError) or isinstance(head_call_error, GatedRepoError): 1402 # Repo not found or gated => let's raise the actual error -> 1403 raise head_call_error 1404 else: 1405 # Otherwise: most likely a connection issue or Hub downtime => let's warn the user

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/file_download.py:1261, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, headers, legacy_cache_layout, endpoint) 1260 try: -> 1261 metadata = get_hf_file_metadata( 1262 url=url, 1263 token=token, 1264 proxies=proxies, 1265 timeout=etag_timeout, 1266 library_name=library_name, 1267 library_version=library_version, 1268 user_agent=user_agent, 1269 ) 1270 except EntryNotFoundError as http_error: 1271 # Cache the non-existence of the file and raise

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:119, in validate_hf_hub_args.._inner_fn(*args, *kwargs) 117 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs) --> 119 return fn(args, **kwargs)

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/file_download.py:1674, in get_hf_file_metadata(url, token, proxies, timeout, library_name, library_version, user_agent, headers) 1673 # Retrieve metadata -> 1674 r = _request_wrapper( 1675 method=\"HEAD\", 1676 url=url, 1677 headers=headers, 1678 allow_redirects=False, 1679 follow_relative_redirects=True, 1680 proxies=proxies, 1681 timeout=timeout, 1682 ) 1683 hf_raise_for_status(r)

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/file_download.py:369, in _request_wrapper(method, url, follow_relative_redirects, params) 368 if follow_relative_redirects: --> 369 response = _request_wrapper( 370 method=method, 371 url=url, 372 follow_relative_redirects=False, 373 params, 374 ) 376 # If redirection, we redirect only relative paths. 377 # This is useful in case of a renamed repository.

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/file_download.py:393, in _request_wrapper(method, url, follow_relative_redirects, params) 392 response = get_session().request(method=method, url=url, params) --> 393 hf_raise_for_status(response) 394 return response

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py:352, in hf_raise_for_status(response, endpoint_name) 344 message = ( 345 f\"{response.status_code} Client Error.\" 346 + \"\ \ \" (...) 350 \" make sure you are authenticated.\" 351 ) --> 352 raise RepositoryNotFoundError(message, response) from e 354 elif response.status_code == 400:

RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-6604e559-788eda1432e6677020219a98;c33cdadc-232b-4b64-9680-57a0bc9c8c99)

Repository Not Found for url: https://huggingface.co/lora_adapters/codellama-bt-adapter/resolve/main/adapter_config.json. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated.

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) Cell In[14], line 2 1 # Load fine-tuned model ----> 2 finetuned_model = PeftModel.from_pretrained(base_model, adapter_id) 3 finetuned_model = finetuned_model.merge_and_unload()

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/peft/peft_model.py:328, in PeftModel.from_pretrained(cls, model, model_id, adapter_name, is_trainable, config, kwargs) 325 # load the config 326 if config is None: 327 config = PEFT_TYPE_TO_CONFIG_MAPPING[ --> 328 PeftConfig._get_peft_type( 329 model_id, 330 subfolder=kwargs.get(\"subfolder\", None), 331 revision=kwargs.get(\"revision\", None), 332 cache_dir=kwargs.get(\"cache_dir\", None), 333 use_auth_token=kwargs.get(\"use_auth_token\", None), 334 token=kwargs.get(\"token\", None), 335 ) 336 ].from_pretrained(model_id, kwargs) 337 elif isinstance(config, PeftConfig): 338 config.inference_mode = not is_trainable

File ~/anaconda3/envs/btgenbotv1/lib/python3.10/site-packages/peft/config.py:203, in PeftConfigMixin._get_peft_type(cls, model_id, hf_hub_download_kwargs) 197 config_file = hf_hub_download( 198 model_id, 199 CONFIG_NAME, 200 hf_hub_download_kwargs, 201 ) 202 except Exception: --> 203 raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{model_id}'\") 205 loaded_attributes = cls.from_json_file(config_file) 206 return loaded_attributes[\"peft_type\"]

ValueError: Can't find 'adapter_config.json' at 'lora_adapters/codellama-bt-adapter'" }

[sea-hot]

"When I ran it again, an error related to the quantization keyword occurred in the output of the code for selecting adapters in the inference.ipynb file of the bt generator folder. I haven't made any other adjustments, just tried rerunning it. I'm wondering if this could be the cause."

[RiccardoIzzo]

The first error related to "adapter_config.json" has been resolved in the latest commit baa218f7c7f1eae0b4f9eab622b785766c79f948 by fixing the path issue. Additionally, for ease of use, we plan to release soon both the dataset and the adapters on HuggingFace. Regarding the error associated with quantization, it is a common issue with the bitsandbytes library from the use of deprecated "load_in_8_bit". We have replaced this with "BitsAndBytesConfig" to solve this. Please try out the updated notebook and provide your feedback.

[RiccardoIzzo]

The adapters are now accessible via HuggingFace, with the inference notebook now directly linking to them.