tian-qing001
commented
5 months ago Hi @Syk-yr, like all other linear attention designs, agent attention cannot support arbitrary attention mask. What type of attention mask would you like to use?
Open Syk-yr opened 6 months ago
tian-qing001
commented
5 months ago Hi @Syk-yr, like all other linear attention designs, agent attention cannot support arbitrary attention mask. What type of attention mask would you like to use?
您好，感谢您提供的思路，我想请问下，如何在agent attention 中加attention mask来屏蔽部分输入？