search

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/
http://h2o.ai
Apache License 2.0
11.19k stars 1.23k forks source link

Executing small model but missing config.json error with microsoft/Phi-3-mini-4k-instruct-gguf #1642

Open harnalashok opened 3 months ago

harnalashok commented 3 months ago

I have installed h2ogpt on ubuntu22.04 as per procedure. But when I run the following command I get an error of missing config.json file. Please let me know how to overcome this error.

The command executed is this: python generate.py --base_model=microsoft/Phi-3-mini-4k-instruct-gguf --max_seq_len=4096

Complete error trace is as below:

python generate.py --base_model=microsoft/Phi-3-mini-4k-instruct-gguf --max_seq_len=4096 /home/ashok/anaconda3/lib/python3.11/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning) soundfile, librosa, and wavio not installed, disabling STT soundfile, librosa, and wavio not installed, disabling TTS Using Model microsoft/phi-3-mini-4k-instruct-gguf load INSTRUCTOR_Transformer max_seq_length 512 Must install DocTR and LangChain installed if enabled DocTR, disabling Starting get_model: microsoft/Phi-3-mini-4k-instruct-gguf /home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py:1132: FutureWarning:resume_downloadis deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, useforce_download=True`. warnings.warn( Not using tokenizer from HuggingFace:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/home/ashok/anaconda3/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1282, in _hf_hub_download_to_cache_dir (url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper hf_raise_for_status(response) File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 315, in hf_raise_for_status raise EntryNotFoundError(message, response) from e huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-664ed9aa-1e37434a62ea5f780ef151e5;bfa58ffc-1dbc-47db-a17b-697e66705b9e)

Entry Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/h2ogpt/src/gen.py", line 2387, in get_config config = AutoConfig.from_pretrained(base_model, token=use_auth_token, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 453, in cached_file raise EnvironmentError( OSError: microsoft/Phi-3-mini-4k-instruct-gguf does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main' for available files. Starting get_model: microsoft/Phi-3-mini-4k-instruct-gguf Not using tokenizer from HuggingFace:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/home/ashok/anaconda3/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1282, in _hf_hub_download_to_cache_dir (url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper hf_raise_for_status(response) File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 315, in hf_raise_for_status raise EntryNotFoundError(message, response) from e huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-664ed9ab-6f335f376f51ddff618ec59f;6faf03e5-aebf-4d97-93b5-fca456c09f8e)

Entry Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/h2ogpt/src/gen.py", line 2387, in get_config config = AutoConfig.from_pretrained(base_model, token=use_auth_token, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 453, in cached_file raise EnvironmentError( OSError: microsoft/Phi-3-mini-4k-instruct-gguf does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main' for available files. Starting get_model: microsoft/Phi-3-mini-4k-instruct-gguf Not using tokenizer from HuggingFace:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/home/ashok/anaconda3/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1282, in _hf_hub_download_to_cache_dir (url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper hf_raise_for_status(response) File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 315, in hf_raise_for_status raise EntryNotFoundError(message, response) from e huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-664ed9ac-443700867f5ca8797a99822d;f76d7a7d-72e2-4a21-957f-24192665affc)

Entry Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/h2ogpt/src/gen.py", line 2387, in get_config config = AutoConfig.from_pretrained(base_model, token=use_auth_token, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 453, in cached_file raise EnvironmentError( OSError: microsoft/Phi-3-mini-4k-instruct-gguf does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main' for available files. Starting get_model: microsoft/Phi-3-mini-4k-instruct-gguf Not using tokenizer from HuggingFace:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/home/ashok/anaconda3/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1282, in _hf_hub_download_to_cache_dir (url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper hf_raise_for_status(response) File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 315, in hf_raise_for_status raise EntryNotFoundError(message, response) from e huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-664ed9ad-059285741225bb7a6fc4dfd0;0d3f6eea-c030-4c14-bcd3-301719c35433)

Entry Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/h2ogpt/src/gen.py", line 2387, in get_config config = AutoConfig.from_pretrained(base_model, token=use_auth_token, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 453, in cached_file raise EnvironmentError( OSError: microsoft/Phi-3-mini-4k-instruct-gguf does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main' for available files. Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/home/ashok/anaconda3/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( ^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1282, in _hf_hub_download_to_cache_dir (url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper hf_raise_for_status(response) File "/home/ashok/anaconda3/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 315, in hf_raise_for_status raise EntryNotFoundError(message, response) from e huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-664ed9ad-2faeddca4207a77937c6b8ba;d77ab411-ccbd-42e2-94d6-f0317b2acd66)

Entry Not Found for url: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/config.json.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ashok/h2ogpt/generate.py", line 20, in entrypoint_main() File "/home/ashok/h2ogpt/generate.py", line 16, in entrypoint_main H2O_Fire(main) File "/home/ashok/h2ogpt/src/utils.py", line 73, in H2O_Fire fire.Fire(component=component, command=args) File "/home/ashok/anaconda3/lib/python3.11/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/h2ogpt/src/gen.py", line 2293, in main model0, tokenizer0, device = get_model_retry(reward_type=False, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/h2ogpt/src/gen.py", line 2652, in get_model_retry model1, tokenizer1, device1 = get_model(kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/ashok/h2ogpt/src/gen.py", line 3318, in get_model return get_hf_model(load_8bit=load_8bit, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/h2ogpt/src/gen.py", line 3413, in get_hfmodel config, , max_seq_len = get_config(base_model, return_model=False, raise_exception=True, config_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/h2ogpt/src/gen.py", line 2387, in get_config config = AutoConfig.from_pretrained(base_model, token=use_auth_token, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( ^^^^^^^^^^^^ File "/home/ashok/anaconda3/lib/python3.11/site-packages/transformers/utils/hub.py", line 453, in cached_file raise EnvironmentError( OSError: microsoft/Phi-3-mini-4k-instruct-gguf does not appear to have a file named config.json. Checkout 'https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/tree/main' for available files. `

pseudotensor commented 3 months ago

Hi, I'll try to clarify the docs, but as in readme.md and in FAQ.md, for new chat template models when using GGUF, you need to specify the tokenizer_base_model.

https://github.com/h2oai/h2ogpt/blob/main/docs/FAQ.md#llama-3-or-other-chat-template-based-models

Also, base_model accepts a few forms of passing urls, TheBloke, etc. for gguf, but not just -gguf, so until I add that, you'll need to use more explicit way of invoking llama.cpp:

e.g. for Phi it would be to use the direct download link like:

python generate.py  --tokenizer_base_model=microsoft/Phi-3-mini-4k-instruct --base_model=llama --llama_cpp_model=https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf  --max_seq_len=4096

or you can download it yourself and place it into llamacpp_path folder.