[Bug]: trainer保存的模型加载失败

PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.

https://paddlenlp.readthedocs.io

Apache License 2.0

12.04k stars 2.93k forks source link

[Bug]: trainer保存的模型加载失败 #4921

Closed zhangyimi closed 1 year ago

zhangyimi commented 1 year ago

软件环境

- paddlepaddle:
- paddlepaddle-gpu:2.4.2 
- paddlenlp: 2.5.1

重复问题

[X] I have searched the existing issues

错误描述

1.使用trainner训练的模型（使用了fasttokenzier）。
2.使用AutoTokenizer.from_pretrained加载出错。

稳定复现步骤 & 代码

tokenizer_config.json如下：

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天，即将关闭。