Topic Proposal: Image Transformers

[hvgazula]

1. Attention is All You Need (absolutely optional)
2. ViT
3. Intriguing properties of ViT
4. Relationship between Self-attention and CNN (will help for the next paper)
5. Why do ViTs work? (ICLR 2022 Spotlight)
6. Please throw in a Transformers + U-nets paper to justify the reading group's name ;)

[hvgazula]

This is my proposal based on a discussion with @adalca in the mailing list

[adalca]

with the caveat that I gully expect them to be 95% hype and 5% engineering: TransUNET UNETR

[hvgazula]

haha..sure..makes me wonder when was the last time the medical imaging field was truly shaken up (by DL) ala U-net style (presumably)

[adalca]

working on it! :D

[hvgazula]

A ConvNet for the 2020s from @adalca