PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.72k stars 2.86k forks source link

【pir save 】Modiy export llama model file in pir mode #8689

Closed xiaoguoguo626807 closed 22 hours ago

xiaoguoguo626807 commented 2 days ago

PR types

Others

PR changes

Others

Description

pcard-67164 修改多处代码支持在pir模式下对llama-2-7b模型导出

  1. 动转静下遇到动态shape 无法导出,需要将paddlenlp/transformers/llama/modeling.py 中关于attn_weights.shape 的判断代码在动转静下跳过。因为动态图运行此处可以拦截错误,动转静不会出现问题。
  2. 当pad_token_id = None 时,PIR下不允许传递给full_like 的value 是none,此处逻辑不完备,generate 函数中会判断如果没有pad_token_id 时将pad_token_id 设置为eos_token_id
  3. PIR下没有print op 且op相关的方法也不同。需要进行分支处理
paddle-bot[bot] commented 2 days ago

Thanks for your contribution!

codecov[bot] commented 2 days ago

Codecov Report

Attention: Patch coverage is 60.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 55.62%. Comparing base (be5bb14) to head (e86f5bc). Report is 3 commits behind head on develop.

Files Patch % Lines
paddlenlp/generation/utils.py 50.00% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #8689 +/- ## ======================================== Coverage 55.61% 55.62% ======================================== Files 620 620 Lines 96965 96991 +26 ======================================== + Hits 53930 53949 +19 - Misses 43035 43042 +7 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.