Consider Token Downsampling as a replacement for ToMe in agentsd

[feffy380]

Token Downsampling (https://arxiv.org/abs/2402.13573) is similar to Token Merging but with less quality loss and a massively simpler implementation. Perhaps combining it with agent attention will improve results of AgentSD? PixArt-Sigma uses a very similar method in their paper (where they call it KV-Compression) The reference implementation is available at https://github.com/ethansmith2000/ImprovedTokenMerge

[tian-qing001]

Hi @feffy380, thanks for your suggestion. We will consider employing Token Downsampling in future work.