Does not support swin-transformer?

facebookresearch / ToMe

A method to increase the speed and lower the memory footprint of existing vision transformers.

Other

931 stars 67 forks source link

Does not support swin-transformer? #15

Closed YoloEliwa closed 1 year ago

YoloEliwa commented 1 year ago

Hello, Thanks for this amazing work! When I use tome, I found it doesn't seem to support swin-transformer.I want to know why it is not supported and whether it be supported in the future?

I am looking forward to your reply!

dbolya commented 1 year ago

Hi, thanks for you interest.

We do not currently support Swin and have no plans on doing so in the future. Swin's shifted window attention is complicated and it would be hard to get much speed benefit because of that.