search

circle-hit / SAPT

Code for ACL 2024 accepted paper titled "SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models"
19 stars 5 forks source link

Pseudo-samples generated #2

Open DaBinGOGOGO opened 2 months ago

DaBinGOGOGO commented 2 months ago

Hi, how is the PET block of pseudo-samples trained and what are the inputs? In the paper, the input of PET block Bref is the initial token[Gen]

circle-hit commented 2 months ago

Hi, how is the PET block of pseudo-samples trained and what are the inputs? In the paper, the input of PET block Bref is the initial token[Gen]

Hi, the generation of pseudo-samples are trained with the following data format:

Input："__gen__" Output："{Instruction} __inp__ {Question}"

where "__gen__" and "__inp__" are newly added special tokens.