search

youweiliang / evit

Python code for ICLR 2022 spotlight paper EViT: Expediting Vision Transformers via Token Reorganizations
Apache License 2.0
162 stars 19 forks source link

Some question #2

Open Andy1621 opened 2 years ago

Andy1621 commented 2 years ago

In the paper, EViT with oracle can obtain higher accuracy when training longer epochs. Similar results are also shown in the DeiT paper. Thus I think the comparison is not very fair. 600 epoch means training 900 epoch in fact.

Besides, does EViT work for small FLOPs? What about 1/4 FLOPs of DeiT (4.6G/4=1.2G)?

image image image
youweiliang commented 2 years ago

Hi, thanks for the questions.

In Table 7, the presence of DeiT is not for comparison with EViT to show which is better, but for easy reference. We would update the table to avoid misunderstanding about the comparison.

By small FLOPs, did you mean using a small keep rate for EViT? Please elaborate. Thanks.

Andy1621 commented 2 years ago

Thanks for your response. Like changing drop_loc and base_keep_rate to reacher MAC 1.2G. In my experiments, most of the methods do not work well in such a setting.