Open ZacharyHertyas opened 4 weeks ago
Hi, thanks for your question and we understand your concern. Intuitively speaking, we think it's okay whether there is a permutation before the reshape. Because our codebase is based on DualPrompt and the above code can be found in https://github.com/JH-LEE-KR/dualprompt-pytorch/blob/master/prompt.py, line 119-122. We suggest you may communicate with the authors directly for help.
https://github.com/JingyangQiao/prompt-gradient-projection/blob/c566b81f1abb19f8350da3fb71149ca9b8061a13/dualprompt-pgp/prompt.py#L120-L123
I want to know if directly performing a reshape here is incorrect. Maybe a
permutation
before thereshape
is needed to correct the position of batch_prompt, as follows?