PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.96k stars 2.91k forks source link

[Bug]: 无法获取OPT模型13b的预训练权重 #8244

Closed runzhech closed 4 months ago

runzhech commented 5 months ago

软件环境

- paddlepaddle:
- paddlepaddle-gpu: 0.0.0.post120
- paddlenlp: 2.7.1.post0

重复问题

错误描述

尝试使用lora的方式精调PaddleNLP/llm/opt模型。文档中显示支持opt-13b的权重,但将llm/opt/lora_argument.json中的model_name_or_path修改为 "facebook/opt-13b"之后,程序报错如下
Traceback (most recent call last):
  File "/workspace/PaddleNLP/llm/finetune_generation.py", line 661, in <module>
    main()
  File "/workspace/PaddleNLP/llm/finetune_generation.py", line 145, in main
    model_config = AutoConfig.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/paddlenlp/transformers/auto/configuration.py", line 201, in from_pretrained
    raise RuntimeError(
RuntimeError: Can't load config for 'facebook/opt-13b'.
Please make sure that 'facebook/opt-13b' is:
- a correct model-identifier of built-in pretrained models,
- or a correct model-identifier of community-contributed pretrained models,
- or the correct path to a directory containing relevant config files.

I0410 06:39:05.874878 128798 process_group_nccl.cc:133] ProcessGroupNCCL destruct 
LAUNCH INFO 2024-04-10 06:39:07,776 Exit code -15

请问该模型是否需要通过其他方式获取?

稳定复现步骤 & 代码

lora_argument.json { "model_name_or_path": "facebook/opt-13b", "dataset_name_or_path": "./data", "output_dir": "./checkpoints/opt_lora_ckpts", "per_device_train_batch_size": 4, "gradient_accumulation_steps": 4, "per_device_eval_batch_size": 8, "eval_accumulation_steps":16, "num_train_epochs": 3, "learning_rate": 3e-04, "warmup_steps": 30, "logging_steps": 1, "evaluation_strategy": "epoch", "save_strategy": "epoch", "src_length": 1024, "max_length": 2048, "fp16": true, "fp16_opt_level": "O2", "do_train": true, "do_eval": true, "disable_tqdm": true, "load_best_model_at_end": true, "eval_with_do_generation": false, "metric_for_best_model": "accuracy", "recompute": true, "save_total_limit": 1, "tensor_parallel_degree": 1, "pipeline_parallel_degree": 1, "lora": true, "zero_padding": false, "use_flash_attention": true }

python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" finetune_generation.py ./opt/lora_argument.json

w5688414 commented 5 months ago

您好,您可以根据需要使用转换脚本把HF模型转换过来,以下是有用的链接:

https://huggingface.co/facebook/opt-13b/tree/main

https://github.com/PaddlePaddle/PaddleNLP/blob/84615ea086add2c662b4a984254e4aae8ff6071c/paddlenlp/transformers/opt/modeling.py#L653

https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#6-pytorch%E6%A8%A1%E5%9E%8B%E6%9D%83%E9%87%8D%E8%BD%AC%E6%8D%A2