search

JingyangQiao / prompt-gradient-projection

Apache License 2.0
22 stars 2 forks source link

Change comment: incorrect operations #4

Open ZacharyHertyas opened 4 weeks ago

ZacharyHertyas commented 4 weeks ago

https://github.com/JingyangQiao/prompt-gradient-projection/blob/c566b81f1abb19f8350da3fb71149ca9b8061a13/dualprompt-pgp/prompt.py#L120-L123

I want to know if directly performing a reshape here is incorrect. Maybe a permutation before the reshape is needed to correct the position of batch_prompt, as follows?

batched_prompt_raw = batched_prompt_raw.permute(0, 2, 1, 3, 4, 5, 6)
batched_prompt = batched_prompt_raw.reshape(
num_layers, batch_size, dual, top_k * length, num_heads, heads_embed_dim
)
JingyangQiao commented 3 weeks ago

Hi, thanks for your question and we understand your concern. Intuitively speaking, we think it's okay whether there is a permutation before the reshape. Because our codebase is based on DualPrompt and the above code can be found in https://github.com/JH-LEE-KR/dualprompt-pytorch/blob/master/prompt.py, line 119-122. We suggest you may communicate with the authors directly for help.