search

LeapLabTHU / Agent-Attention

Official repository of Agent Attention (ECCV2024)
473 stars 35 forks source link

How to apply AgentAttention to the input of seq_length=320 #39

Closed xyl-507 closed 1 month ago

xyl-507 commented 1 month ago

Hi! @tian-qing001 Thank you for your outstanding contributions. I want to FOLLOW your work. AgentAttention needs to square the input sequence length to get h and w for subsequent agent token. https://github.com/LeapLabTHU/Agent-Attention/blob/ccd29e2f1e8c84d0aba30eee0e9a3c6dfdbda42d/agent_transformer/models/agent_deit.py#L225 However, in the process of applying to my own framework, I found that the input sequence length is 320, and there will be a problem after squaring, is there a solution for this? Thanks in advance!

tian-qing001 commented 1 month ago

Hi @xyl-507, thanks for recognizing our work haha~ The core idea of our implementation is to obtain the agent tokens from queries $Q$ through certain downsampling method like 2D-Pooling. If 2D-Pooling is not suitable for your model, you can consider other appropriate approach to get the agent tokens.

xyl-507 commented 1 month ago

Thanks！I'll try.