search

THUDM / P-tuning

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.
MIT License
912 stars 111 forks source link

A problem about the prompt #2

Closed logoutAgain closed 3 years ago

logoutAgain commented 3 years ago

Is the input of bi-directional model randomly initialized in the process of p-tuning? Or It's the embedding of template. The pseudo prompts in Figure 2(b) seems to indicate that the model needs to use template embedding as input. I'm a little confused about this description.

Xiao9905 commented 3 years ago

The pseudo prompts are randomly initialized, which serve as the placeholders. And to think from another perspective, this is because we can only randomly initialize the prompt encoder.

Xiao9905 commented 3 years ago

Thanks for your comment, and we will better clarify it.

logoutAgain commented 3 years ago

I get it. This makes sense. Thank you for the reply.