search

xxxnell / how-do-vits-work

(ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"
https://arxiv.org/abs/2202.06709
Apache License 2.0
806 stars 79 forks source link

ViT vs ResNet: Did you use SAM optimizer? #28

Closed quannguyen268 closed 1 year ago

quannguyen268 commented 1 year ago

In section 1.1 Related works, I see that you have a comparision between loss landscape of ResNet and ViT. Did you use SAM optimizer while training both models at this stage of comparing or not ? image

xxxnell commented 1 year ago

Hi @quannguyen268, thank you for reaching out.

We do not use SAM optimizer for our experiments.