PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
12.09k stars 2.93k forks source link

paddle在GPU上训练保存的模型,如何在CPU上加载推理使用。 #2047

Closed wpfnlp closed 1 year ago

wpfnlp commented 2 years ago

您好,第一次用paddle,有如下问题想问下: 1,在GPU上训练:model.save_pretrained("./checkpoint')导出的模型,如何在CPU上推理使用? 2,在GPU上训练:paddle.save(model.state_dict(), save_param_path)保存的参数,直接在在CPU上初始化模型。直接load参数就可以在CPU上推理使用吧? 3,我现在想把GPU上训练的模型,在保证精度一致的情况下想在GPU或CPU上做模型轻量化、加速推理操作,请问怎么操作呢?有没示例代码?

谢谢!

ZeyuChen commented 2 years ago

我们将在下个月中v 2.3正式版将这部份文档提供,包含轻量化的cpu gpu部署和服务话部署。 @joey12300 @LiuChiachi @heliqi

wpfnlp commented 2 years ago

您好,因为我们参加了算法比赛,用的是paddle,我们需要在本月8号之前提交镜像,最终评测要在1,CPU模式下:内存占用和推理速度;2,GPU模式下:内存占用和推理速度。两种模式下的综合指标。

所以需要尽快拿到具体的paddle方案来测试下。谢谢。

ZeyuChen commented 2 years ago

您好,因为我们参加了算法比赛,用的是paddle,我们需要在本月8号之前提交镜像,最终评测要在1,CPU模式下:内存占用和推理速度;2,GPU模式下:内存占用和推理速度。两种模式下的综合指标。

所以需要尽快拿到具体的paddle方案来测试下。谢谢。

方便说下是什么比赛吗?以及另外目前你选择的模型是什么?我们看看能不能真好提供匹配模型的cpu gpu推理部署方案。

wpfnlp commented 2 years ago

https://2022.yingjiesheng.com/youwinUploadSys/#/ 预训练用的这个:model = ppnlp.transformers.RobertaForSequenceClassification.from_pretrained("roberta-wwm-ext-large", num_classes=4) 。

wpfnlp commented 2 years ago

您好, @ZeyuChen 部署方案上有文档可以给到吗?谢谢!

ZeyuChen commented 2 years ago

您好, @ZeyuChen 部署方案上有文档可以给到吗?谢谢!

https://github.com/PaddlePaddle/PaddleNLP/blob/develop/examples/text_classification/pretrained_models/export_model.py

@wpfnlp 请尝试下这个导出脚本代码进行部署吧,RoBERTa的结构和bert是一样的,这个脚本应该可以使用。

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

wpfnlp commented 1 year ago

您的邮件已收到!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

wpfnlp commented 1 year ago

您的邮件已收到!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

wpfnlp commented 1 year ago

您的邮件已收到!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

wpfnlp commented 1 year ago

您的邮件已收到!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。