Hi @KingSpencer @zizhaozhang , I have a doubt regarding prompt pool initialization in every new task, inside train_continual.py > train_and_evaluate_per_task(). Why do you transfer previously learned prompt keys to the new prompt parameters in prompt pool and prompt key (lines 550-584)?
Based on my understanding from the paper, the prompt pool and prompt key are shared across tasks and the method learns to select the relevant ones using query function-key matching. So if a subsequent incremental task is initializing prompts from the previous step, why are the prompts being shifted? Are new prompts being added at every step or am I missing something?
Hi @KingSpencer @zizhaozhang , I have a doubt regarding prompt pool initialization in every new task, inside
train_continual.py
>train_and_evaluate_per_task()
. Why do you transfer previously learned prompt keys to the new prompt parameters in prompt pool and prompt key (lines 550-584)?Based on my understanding from the paper, the prompt pool and prompt key are shared across tasks and the method learns to select the relevant ones using query function-key matching. So if a subsequent incremental task is initializing prompts from the previous step, why are the prompts being shifted? Are new prompts being added at every step or am I missing something?
Any help will be appreciated.