search

yoxu515 / aot-benchmark

An efficient modular implementation of Associating Objects with Transformers for Video Object Segmentation in PyTorch
BSD 3-Clause "New" or "Revised" License
600 stars 108 forks source link

SDPA Attention #76

Open bhack opened 6 months ago

bhack commented 6 months ago

@z-x-yang Any plan to reformulate one of the available attentions on using the new official pytorch SDPA? https://pytorch.org/blog/pytorch2-2/

I think that we will have a lot of speed-up and resource optimization with the underline flashattentionv2 implementation.