search

baichuan-inc / Baichuan2

A series of large language models developed by Baichuan Intelligent Technology
https://huggingface.co/baichuan-inc
Apache License 2.0
4.03k stars 289 forks source link

调用微调模型进行推理，报错OSError: ./models/augment-step2-8e-4 does not appear to have a file named config.json. #335

Closed LuckyGlass closed 6 months ago

LuckyGlass commented 6 months ago 
Traceback (most recent call last):
  File "/home/sts/bigai/code/tongos/zhsf_wh/zhsf_demo/demo/demo.py", line 10, in <module>
    model = PKULaw('KEYS')
            ^^^^^^^^^^^^^^
  File "/home/sts/bigai/code/tongos/zhsf_wh/zhsf_demo/demo/launch.py", line 24, in __init__
    self.model = BaichuanForCausalLM.from_pretrained(model_path, device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sts/bigai/code/tongos/zhsf_wh/zhsf_demo/demo/temp_py/modeling_baichuan.py", line 577, in from_pretrained
    config, model_kwargs = cls.config_class.from_pretrained(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sts/anaconda3/envs/zhsf-wh/lib/python3.11/site-packages/transformers/configuration_utils.py", line 591, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sts/anaconda3/envs/zhsf-wh/lib/python3.11/site-packages/transformers/configuration_utils.py", line 620, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sts/anaconda3/envs/zhsf-wh/lib/python3.11/site-packages/transformers/configuration_utils.py", line 675, in _get_config_dict
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/home/sts/anaconda3/envs/zhsf-wh/lib/python3.11/site-packages/transformers/utils/hub.py", line 400, in cached_file
    raise EnvironmentError(
OSError: ./models/augment-step2-8e-4 does not appear to have a file named config.json. Checkout 'https://huggingface.co/./models/augment-step2-8e-4/main' for available files.

具体情况如下：我在一台机器上用提供的fine-tune.py微调成功了一个模型，并保存在augment-step2-8e-4文件夹内，在该机器上可以正常进行推理。将这个augment-step2-8e-4和baichuan2-7b-chat的全部文件拷贝到另一台机器上，再进行推理，报错如上。 文件夹内的确没有config.json，但在原机器上是可以正常推理的。

LuckyGlass commented 6 months ago

问题已经解决，需要安装peft才能调用微调后的模型。 可以使用 tranformers.utils.is_peft_available() 检测peft是否可用，只有peft可用时，AutoModelForCausalLM 才能正确识别微调模型文件夹中的adapter等文件。