search

PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.71k stars 2.86k forks source link

[Bug]: UNIMO模型的resize_token_embeddings方法不会修改decoder的vocab_size，导致报错 #8651

Open JasonCZH4 opened 4 days ago

JasonCZH4 commented 4 days ago

软件环境

- paddlepaddle: 2.5.2
- paddlepaddle-gpu: 2.5.2
- paddlenlp: 2.8.0

重复问题

错误描述

UNIMO模型的resize_token_embeddings方法不会修改decoder的vocab_size，导致input_embeddings_size和output_embeddings_size没法对齐

稳定复现步骤 & 代码

tokenizer = UNIMOTokenizer.from_pretrained('./unimo-text-1.0-large')
model.resize_token_embeddings(len(tokenizer)) 
print(model.get_input_embeddings().weight.shape, model.lm_head.weight.shape)
JasonCZH4 commented 4 days ago

GPT2模型也有类似问题，但是他已经被修复了，参考link，我使用类似方法修改unimo/modeling.py后可以修复，后续会提个PR。