ssbuild / chatglm3_finetuning

Apache License 2.0
37 stars 2 forks source link

Repo not found? #1

Closed Jeru2023 closed 11 months ago

Jeru2023 commented 12 months ago

$ bash train_lora.sh -m dataset INFO:lightning_fabric.utilities.seed:Seed set to 42 Traceback (most recent call last): File "D:\dev\chatglm3_finetuning\datautils.py", line 159, in tokenizer, config, ,_ = dataHelper.load_tokenizer_and_config(tokenizer_class_name=ChatGLMTokenizer, File "D:\conda_envs\ai\lib\site-packages\deep_training\data_helper\data_helper.py", line 190, in load_tokenizer_and_config tokenizer = load_tokenizer(tokenizer_name=tokenizer_name or model_args.tokenizer_name, File "D:\conda_envs\ai\lib\site-packages\numpy_io\pytorch_loader\tokenizer_config_helper.py", line 37, in load_tokenizer tokenizer = class_name.from_pretrained(tokenizer_name or model_name_or_path, **tokenizer_kwargs) File "D:\conda_envs\ai\lib\site-packages\transformers\tokenization_utils_base.py", line 1784, in from_pretrained resolved_vocab_files[file_id] = cached_file( File "D:\conda_envs\ai\lib\site-packages\transformers\utils\hub.py", line 417, in cached_file resolved_file = hf_hub_download( File "D:\conda_envs\ai\lib\site-packages\huggingface_hub\utils_validators.py", line 110, in _inner_fn validate_repo_id(arg_value) File "D:\conda_envs\ai\lib\site-packages\huggingface_hub\utils_validators.py", line 158, in validate_repo_id raise HFValidationError( huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/nlp/pre_models/torch/chatglm3/chatglm3-6b'. Use repo_type argument if needed.

ssbuild commented 12 months ago
  1. 离线下载权重到本地路径, 并且配置下常量 https://github.com/ssbuild/chatglm3_finetuning/blob/dev/config/constant_map.py
  2. 在对应scripts/config/train_{}.yaml 修改对应的global_args model_name。
Jeru2023 commented 12 months ago

Perfectly resolved! tks.