search

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
https://optimalscale.github.io/LMFlow/
Apache License 2.0
8.23k stars 823 forks source link

OSError: output_models/finetune_with_lora/checkpoint-5000 does not appear to have a file named config.json. #104

Closed ChrisXULC closed 1 year ago

ChrisXULC commented 1 year ago

LoRA does not support RAM optimized load currently. Automatically use original load instead. Traceback (most recent call last): File "/root/LMFlow/examples/evaluate.py", line 33, in model = AutoModel.get_model(model_args, tune_strategy='none', ds_config=ds_config) File "/root/LMFlow/examples/lmflow/models/auto_model.py", line 14, in get_model return HFDecoderModel(model_args, *args, kwargs) File "/root/LMFlow/examples/lmflow/models/hf_decoder_model.py", line 213, in init self.backend_model = AutoModelForCausalLM.from_pretrained( File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/transformers/models/auto/configuration_auto.py", line 896, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/transformers/configuration_utils.py", line 573, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/transformers/configuration_utils.py", line 628, in _get_config_dict resolved_config_file = cached_file( File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/transformers/utils/hub.py", line 380, in cached_file raise EnvironmentError( OSError: output_models/finetune_with_lora/checkpoint-5000 does not appear to have a file named config.json. Checkout 'https://huggingface.co/output_models/finetune_with_lora/checkpoint-5000/None' for available files. [2023-04-04 14:37:35,087] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 1580 [2023-04-04 14:37:35,088] [ERROR] [launch.py:324:sigkill_handler] ['/root/anaconda3/envs/lmflow/bin/python', '-u', 'examples/evaluate.py', '--local_rank=0', '--answer_type', 'text', '--model_name_or_path', 'output_models/finetune_with_lora/checkpoint-5000', '--lora_model_path', 'output_models/finetune_with_lora', '--dataset_path', 'data/alpaca/test', '--prompt_structure', 'Input: {input}', '--deepspeed', 'examples/ds_config.json'] exits with return code = 1, model path那边不是微调过后的checkpoint 的path吗

research4pan commented 1 year ago

Thanks for your attention! For --model_name_or_path you may need to specify the original model, such as facebook/galactica-1.3b. Thanks 😄

ChrisXULC commented 1 year ago

Thanks for your attention! For --model_name_or_path you may need to specify the original model, such as facebook/galactica-1.3b. Thanks 😄

Thanks for your reply. I have another question, if I want to fine tune with a model saved in my computer, what should be filled in --model_name_or_path in file run_finetune.sh?

research4pan commented 1 year ago

If your model is trained in normally manner, then you may directly fill --model_name_or_path with the path to your checkpoint directory (which contains the config.json file). If your model is trained with LoRA, then you can specify --model_name_or_path with the original model you trained, either a normally trained checkpoint or a huggingface model name, along with --lora_model_path to your LoRA model checkpoint.

This difference is caused by the internal property of LoRA, which just stores model difference instead of model itself. Hope that answers your question~ 😄

BA2Ops commented 1 year ago

Hello. I'm not very familiar with the use of LMFlow. I'm following the documentation of LMFlow on google colab trying to initialize the environment. Execute without parameters “! /content/LMFlow/scripts/run_chatbot.sh” can enter the chatbot command prompt normally. I got the existing output model using the following command

%cd output_models !bash download.sh all image

How do I need to set the parameters to run the chatbot with these models? I get the following error when I try the command ! /content/LMFlow/scripts/run_chatbot.sh /content/LMFlow/output_models/llama30b-loar-170k

[2023-04-09 03:30:06,863] [WARNING] [runner.py:186:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. Detected CUDA_VISIBLE_DEVICES=0: setting --include=localhost:0 [2023-04-09 03:30:06,876] [INFO] [runner.py:550:main] cmd = /usr/bin/python3 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMF19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None examples/chatbot.py --deepspeed configs/ds_config_chatbot.json --model_name_or_path llama30b --lora_model_path loar-170k [2023-04-09 03:30:09,280] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NCCL_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE_NAME=libnccl2 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:142:main] WORLD INFO DICT: {'localhost': [0]} [2023-04-09 03:30:09,281] [INFO] [launch.py:148:main] nnodes=1, num_local_procs=1, node_rank=0 [2023-04-09 03:30:09,281] [INFO] [launch.py:161:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0]}) [2023-04-09 03:30:09,281] [INFO] [launch.py:162:main] dist_world_size=1 [2023-04-09 03:30:09,281] [INFO] [launch.py:164:main] Setting CUDA_VISIBLE_DEVICES=0 2023-04-09 03:30:13.883796: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_errors.py:259 │ │ in hf_raise_for_status │ │ │ │ 256 │ │ │ 257 │ """ │ │ 258 │ try: │ │ ❱ 259 │ │ response.raise_for_status() │ │ 260 │ except HTTPError as e: │ │ 261 │ │ error_code = response.headers.get("X-Error-Code") │ │ 262 │ │ │ │ /usr/local/lib/python3.9/dist-packages/requests/models.py:960 in │ │ raise_for_status │ │ │ │ 957 │ │ │ http_error_msg = u'%s Server Error: %s for url: %s' % (sel │ │ 958 │ │ │ │ 959 │ │ if http_error_msg: │ │ ❱ 960 │ │ │ raise HTTPError(http_error_msg, response=self) │ │ 961 │ │ │ 962 │ def close(self): │ │ 963 │ │ """Releases the connection back to the pool. Once this method │ ╰──────────────────────────────────────────────────────────────────────────────╯ HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/llama30b/resolve/main/config.json

The above exception was the direct cause of the following exception:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py:409 in │ │ cached_file │ │ │ │ 406 │ user_agent = http_user_agent(user_agent) │ │ 407 │ try: │ │ 408 │ │ # Load from URL or cache if already cached │ │ ❱ 409 │ │ resolved_file = hf_hub_download( │ │ 410 │ │ │ path_or_repo_id, │ │ 411 │ │ │ filename, │ │ 412 │ │ │ subfolder=None if len(subfolder) == 0 else subfolder, │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_validators.py: │ │ 120 in _inner_fn │ │ │ │ 117 │ │ if check_use_auth_token: │ │ 118 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__na │ │ 119 │ │ │ │ ❱ 120 │ │ return fn(*args, *kwargs) │ │ 121 │ │ │ 122 │ return _inner_fn # type: ignore │ │ 123 │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/file_download.py:1166 │ │ in hf_hub_download │ │ │ │ 1163 │ if not local_files_only: │ │ 1164 │ │ try: │ │ 1165 │ │ │ try: │ │ ❱ 1166 │ │ │ │ metadata = get_hf_file_metadata( │ │ 1167 │ │ │ │ │ url=url, │ │ 1168 │ │ │ │ │ token=token, │ │ 1169 │ │ │ │ │ proxies=proxies, │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_validators.py: │ │ 120 in _inner_fn │ │ │ │ 117 │ │ if check_use_auth_token: │ │ 118 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__na │ │ 119 │ │ │ │ ❱ 120 │ │ return fn(args, **kwargs) │ │ 121 │ │ │ 122 │ return _inner_fn # type: ignore │ │ 123 │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/file_download.py:1507 │ │ in get_hf_file_metadata │ │ │ │ 1504 │ │ proxies=proxies, │ │ 1505 │ │ timeout=timeout, │ │ 1506 │ ) │ │ ❱ 1507 │ hf_raise_for_status(r) │ │ 1508 │ │ │ 1509 │ # Return │ │ 1510 │ return HfFileMetadata( │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_errors.py:291 │ │ in hf_raise_for_status │ │ │ │ 288 │ │ │ │ " repo_type.\nIf you are trying to access a private │ │ 289 │ │ │ │ " make sure you are authenticated." │ │ 290 │ │ │ ) │ │ ❱ 291 │ │ │ raise RepositoryNotFoundError(message, response) from e │ │ 292 │ │ │ │ 293 │ │ elif response.status_code == 400: │ │ 294 │ │ │ message = ( │ ╰──────────────────────────────────────────────────────────────────────────────╯ RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-64323148-394188a71d0657a41227ca94)

Repository Not Found for url: https://huggingface.co/llama30b/resolve/main/config.json. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated. Invalid username or password.

During handling of the above exception, another exception occurred:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /content/LMFlow/examples/chatbot.py:143 in │ │ │ │ 140 │ │ 141 │ │ 142 if name == "main": │ │ ❱ 143 │ main() │ │ 144 │ │ │ │ /content/LMFlow/examples/chatbot.py:68 in main │ │ │ │ 65 │ with open (pipeline_args.deepspeed, "r") as f: │ │ 66 │ │ ds_config = json.load(f) │ │ 67 │ │ │ ❱ 68 │ model = AutoModel.get_model( │ │ 69 │ │ model_args, │ │ 70 │ │ tune_strategy='none', │ │ 71 │ │ ds_config=ds_config, │ │ │ │ /content/LMFlow/src/lmflow/models/auto_model.py:14 in get_model │ │ │ │ 11 │ @classmethod │ │ 12 │ def get_model(self, model_args, *args, *kwargs): │ │ 13 │ │ # TODO (add new models) │ │ ❱ 14 │ │ return HFDecoderModel(model_args, args, **kwargs) │ │ 15 │ │ │ │ /content/LMFlow/src/lmflow/models/hf_decoder_model.py:220 in init │ │ │ │ 217 │ │ │ │ │ │ "LoRA does not support RAM optimized load curr │ │ 218 │ │ │ │ │ │ " Automatically use original load instead." │ │ 219 │ │ │ │ │ ) │ │ ❱ 220 │ │ │ │ self.backend_model = AutoModelForCausalLM.from_pretrai │ │ 221 │ │ │ │ │ model_args.model_name_or_path, │ │ 222 │ │ │ │ ) │ │ 223 │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/models/auto/auto_factory │ │ .py:441 in from_pretrained │ │ │ │ 438 │ │ │ if kwargs_copy.get("torchdtype", None) == "auto": │ │ 439 │ │ │ │ = kwargs_copy.pop("torch_dtype") │ │ 440 │ │ │ │ │ ❱ 441 │ │ │ config, kwargs = AutoConfig.from_pretrained( │ │ 442 │ │ │ │ pretrained_model_name_or_path, │ │ 443 │ │ │ │ return_unused_kwargs=True, │ │ 444 │ │ │ │ trust_remote_code=trust_remote_code, │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/models/auto/configuratio │ │ n_auto.py:908 in from_pretrained │ │ │ │ 905 │ │ kwargs["_from_auto"] = True │ │ 906 │ │ kwargs["name_or_path"] = pretrained_model_name_or_path │ │ 907 │ │ trust_remote_code = kwargs.pop("trust_remote_code", False) │ │ ❱ 908 │ │ config_dict, unused_kwargs = PretrainedConfig.get_config_dict( │ │ 909 │ │ if "auto_map" in config_dict and "AutoConfig" in config_dict[" │ │ 910 │ │ │ if not trust_remote_code: │ │ 911 │ │ │ │ raise ValueError( │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/configuration_utils.py:5 │ │ 73 in get_config_dict │ │ │ │ 570 │ │ """ │ │ 571 │ │ original_kwargs = copy.deepcopy(kwargs) │ │ 572 │ │ # Get config dict associated with the base config file │ │ ❱ 573 │ │ config_dict, kwargs = cls._get_config_dict(pretrained_model_na │ │ 574 │ │ if "_commit_hash" in config_dict: │ │ 575 │ │ │ original_kwargs["_commit_hash"] = config_dict["_commit_has │ │ 576 │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/configuration_utils.py:6 │ │ 28 in _get_config_dict │ │ │ │ 625 │ │ │ │ │ 626 │ │ │ try: │ │ 627 │ │ │ │ # Load from local folder or from cache or download fro │ │ ❱ 628 │ │ │ │ resolved_config_file = cached_file( │ │ 629 │ │ │ │ │ pretrained_model_name_or_path, │ │ 630 │ │ │ │ │ configuration_file, │ │ 631 │ │ │ │ │ cache_dir=cache_dir, │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py:424 in │ │ cached_file │ │ │ │ 421 │ │ ) │ │ 422 │ │ │ 423 │ except RepositoryNotFoundError: │ │ ❱ 424 │ │ raise EnvironmentError( │ │ 425 │ │ │ f"{path_or_repo_id} is not a local folder and is not a va │ │ 426 │ │ │ "listed on 'https://huggingface.co/models'\nIf this is a │ │ 427 │ │ │ "pass a token having permission to this repo with use_au │ ╰──────────────────────────────────────────────────────────────────────────────╯ OSError: llama30b is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo withuse_auth_tokenor log in withhuggingface-cli loginand pass use_auth_token=True`. [2023-04-09 03:30:18,302] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 8983 [2023-04-09 03:30:18,303] [ERROR] [launch.py:324:sigkill_handler] ['/usr/bin/python3', '-u', 'examples/chatbot.py', '--local_rank=0', '--deepspeed', 'configs/ds_config_chatbot.json', '--model_name_or_path', 'llama30b', '--lora_model_path', 'loar-170k'] exits with return code = 1

research4pan commented 1 year ago

Hello. I'm not very familiar with the use of LMFlow. I'm following the documentation of LMFlow on google colab trying to initialize the environment. Execute without parameters “! /content/LMFlow/scripts/run_chatbot.sh” can enter the chatbot command prompt normally. I got the existing output model using the following command

%cd output_models !bash download.sh all image

How do I need to set the parameters to run the chatbot with these models? I get the following error when I try the command ! /content/LMFlow/scripts/run_chatbot.sh /content/LMFlow/output_models/llama30b-loar-170k

[2023-04-09 03:30:06,863] [WARNING] [runner.py:186:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. Detected CUDA_VISIBLE_DEVICES=0: setting --include=localhost:0 [2023-04-09 03:30:06,876] [INFO] [runner.py:550:main] cmd = /usr/bin/python3 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMF19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None examples/chatbot.py --deepspeed configs/ds_config_chatbot.json --model_name_or_path llama30b --lora_model_path loar-170k [2023-04-09 03:30:09,280] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NCCL_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE_NAME=libnccl2 [2023-04-09 03:30:09,281] [INFO] [launch.py:135:main] 0 NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 [2023-04-09 03:30:09,281] [INFO] [launch.py:142:main] WORLD INFO DICT: {'localhost': [0]} [2023-04-09 03:30:09,281] [INFO] [launch.py:148:main] nnodes=1, num_local_procs=1, node_rank=0 [2023-04-09 03:30:09,281] [INFO] [launch.py:161:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0]}) [2023-04-09 03:30:09,281] [INFO] [launch.py:162:main] dist_world_size=1 [2023-04-09 03:30:09,281] [INFO] [launch.py:164:main] Setting CUDA_VISIBLE_DEVICES=0 2023-04-09 03:30:13.883796: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_errors.py:259 │ │ in hf_raise_for_status │ │ │ │ 256 │ │ │ 257 │ """ │ │ 258 │ try: │ │ ❱ 259 │ │ response.raise_for_status() │ │ 260 │ except HTTPError as e: │ │ 261 │ │ error_code = response.headers.get("X-Error-Code") │ │ 262 │ │ │ │ /usr/local/lib/python3.9/dist-packages/requests/models.py:960 in │ │ raise_for_status │ │ │ │ 957 │ │ │ http_error_msg = u'%s Server Error: %s for url: %s' % (sel │ │ 958 │ │ │ │ 959 │ │ if http_error_msg: │ │ ❱ 960 │ │ │ raise HTTPError(http_error_msg, response=self) │ │ 961 │ │ │ 962 │ def close(self): │ │ 963 │ │ """Releases the connection back to the pool. Once this method │ ╰──────────────────────────────────────────────────────────────────────────────╯ HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/llama30b/resolve/main/config.json

The above exception was the direct cause of the following exception:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py:409 in │ │ cached_file │ │ │ │ 406 │ user_agent = http_user_agent(user_agent) │ │ 407 │ try: │ │ 408 │ │ # Load from URL or cache if already cached │ │ ❱ 409 │ │ resolved_file = hf_hub_download( │ │ 410 │ │ │ path_or_repo_id, │ │ 411 │ │ │ filename, │ │ 412 │ │ │ subfolder=None if len(subfolder) == 0 else subfolder, │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_validators.py: │ │ 120 in _inner_fn │ │ │ │ 117 │ │ if check_use_auth_token: │ │ 118 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__na │ │ 119 │ │ │ │ ❱ 120 │ │ return fn(*args, *kwargs) │ │ 121 │ │ │ 122 │ return _inner_fn # type: ignore │ │ 123 │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/file_download.py:1166 │ │ in hf_hub_download │ │ │ │ 1163 │ if not local_files_only: │ │ 1164 │ │ try: │ │ 1165 │ │ │ try: │ │ ❱ 1166 │ │ │ │ metadata = get_hf_file_metadata( │ │ 1167 │ │ │ │ │ url=url, │ │ 1168 │ │ │ │ │ token=token, │ │ 1169 │ │ │ │ │ proxies=proxies, │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_validators.py: │ │ 120 in _inner_fn │ │ │ │ 117 │ │ if check_use_auth_token: │ │ 118 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__na │ │ 119 │ │ │ │ ❱ 120 │ │ return fn(args, **kwargs) │ │ 121 │ │ │ 122 │ return _inner_fn # type: ignore │ │ 123 │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/file_download.py:1507 │ │ in get_hf_file_metadata │ │ │ │ 1504 │ │ proxies=proxies, │ │ 1505 │ │ timeout=timeout, │ │ 1506 │ ) │ │ ❱ 1507 │ hf_raise_for_status(r) │ │ 1508 │ │ │ 1509 │ # Return │ │ 1510 │ return HfFileMetadata( │ │ │ │ /usr/local/lib/python3.9/dist-packages/huggingface_hub/utils/_errors.py:291 │ │ in hf_raise_for_status │ │ │ │ 288 │ │ │ │ " repo_type.\nIf you are trying to access a private │ │ 289 │ │ │ │ " make sure you are authenticated." │ │ 290 │ │ │ ) │ │ ❱ 291 │ │ │ raise RepositoryNotFoundError(message, response) from e │ │ 292 │ │ │ │ 293 │ │ elif response.status_code == 400: │ │ 294 │ │ │ message = ( │ ╰──────────────────────────────────────────────────────────────────────────────╯ RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-64323148-394188a71d0657a41227ca94)

Repository Not Found for url: https://huggingface.co/llama30b/resolve/main/config.json. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated. Invalid username or password.

During handling of the above exception, another exception occurred:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /content/LMFlow/examples/chatbot.py:143 in │ │ │ │ 140 │ │ 141 │ │ 142 if name == "main": │ │ ❱ 143 │ main() │ │ 144 │ │ │ │ /content/LMFlow/examples/chatbot.py:68 in main │ │ │ │ 65 │ with open (pipeline_args.deepspeed, "r") as f: │ │ 66 │ │ ds_config = json.load(f) │ │ 67 │ │ │ ❱ 68 │ model = AutoModel.get_model( │ │ 69 │ │ model_args, │ │ 70 │ │ tune_strategy='none', │ │ 71 │ │ ds_config=ds_config, │ │ │ │ /content/LMFlow/src/lmflow/models/auto_model.py:14 in get_model │ │ │ │ 11 │ @classmethod │ │ 12 │ def get_model(self, model_args, *args, *kwargs): │ │ 13 │ │ # TODO (add new models) │ │ ❱ 14 │ │ return HFDecoderModel(model_args, args, kwargs) │ │ 15 │ │ │ │ /content/LMFlow/src/lmflow/models/hf_decoder_model.py:220 in init** │ │ │ │ 217 │ │ │ │ │ │ "LoRA does not support RAM optimized load curr │ │ 218 │ │ │ │ │ │ " Automatically use original load instead." │ │ 219 │ │ │ │ │ ) │ │ ❱ 220 │ │ │ │ self.backend_model = AutoModelForCausalLM.from_pretrai │ │ 221 │ │ │ │ │ model_args.model_name_or_path, │ │ 222 │ │ │ │ ) │ │ 223 │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/models/auto/auto_factory │ │ .py:441 in from_pretrained │ │ │ │ 438 │ │ │ if kwargs_copy.get("torchdtype", None) == "auto": │ │ 439 │ │ │ │ = kwargs_copy.pop("torch_dtype") │ │ 440 │ │ │ │ │ ❱ 441 │ │ │ config, kwargs = AutoConfig.from_pretrained( │ │ 442 │ │ │ │ pretrained_model_name_or_path, │ │ 443 │ │ │ │ return_unused_kwargs=True, │ │ 444 │ │ │ │ trust_remote_code=trust_remote_code, │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/models/auto/configuratio │ │ n_auto.py:908 in from_pretrained │ │ │ │ 905 │ │ kwargs["_from_auto"] = True │ │ 906 │ │ kwargs["name_or_path"] = pretrained_model_name_or_path │ │ 907 │ │ trust_remote_code = kwargs.pop("trust_remote_code", False) │ │ ❱ 908 │ │ config_dict, unused_kwargs = PretrainedConfig.get_config_dict( │ │ 909 │ │ if "auto_map" in config_dict and "AutoConfig" in config_dict[" │ │ 910 │ │ │ if not trust_remote_code: │ │ 911 │ │ │ │ raise ValueError( │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/configuration_utils.py:5 │ │ 73 in get_config_dict │ │ │ │ 570 │ │ """ │ │ 571 │ │ original_kwargs = copy.deepcopy(kwargs) │ │ 572 │ │ # Get config dict associated with the base config file │ │ ❱ 573 │ │ config_dict, kwargs = cls._get_config_dict(pretrained_model_na │ │ 574 │ │ if "_commit_hash" in config_dict: │ │ 575 │ │ │ original_kwargs["_commit_hash"] = config_dict["_commit_has │ │ 576 │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/configuration_utils.py:6 │ │ 28 in _get_config_dict │ │ │ │ 625 │ │ │ │ │ 626 │ │ │ try: │ │ 627 │ │ │ │ # Load from local folder or from cache or download fro │ │ ❱ 628 │ │ │ │ resolved_config_file = cached_file( │ │ 629 │ │ │ │ │ pretrained_model_name_or_path, │ │ 630 │ │ │ │ │ configuration_file, │ │ 631 │ │ │ │ │ cache_dir=cache_dir, │ │ │ │ /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py:424 in │ │ cached_file │ │ │ │ 421 │ │ ) │ │ 422 │ │ │ 423 │ except RepositoryNotFoundError: │ │ ❱ 424 │ │ raise EnvironmentError( │ │ 425 │ │ │ f"{path_or_repo_id} is not a local folder and is not a va │ │ 426 │ │ │ "listed on 'https://huggingface.co/models'\nIf this is a │ │ 427 │ │ │ "pass a token having permission to this repo with use_au │ ╰──────────────────────────────────────────────────────────────────────────────╯ OSError: llama30b is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo withuse_auth_tokenor log in withhuggingface-cli loginand passuse_auth_token=True`. [2023-04-09 03:30:18,302] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 8983 [2023-04-09 03:30:18,303] [ERROR] [launch.py:324:sigkill_handler] ['/usr/bin/python3', '-u', 'examples/chatbot.py', '--local_rank=0', '--deepspeed', 'configs/ds_config_chatbot.json', '--model_name_or_path', 'llama30b', '--lora_model_path', 'loar-170k'] exits with return code = 1

Thanks for your attention! You may run something like ! /content/LMFlow/scripts/run_chatbot.sh pinkmanlove/llama-33b-hf /content/LMFlow/output_models/llama30b-loar-170k, since for LoRA models, the original llama model has to be specified as well. Thanks!

shizhediao commented 1 year ago

This issue has been marked as stale because it has not had recent activity. If you think this still needs to be addressed please feel free to reopen this issue. Thanks!