-
### 🚀 The feature
As the title suggest, add the data augmentation from https://arxiv.org/abs/2204.07118
### Motivation, pitch
This seems to be a simple recipe with good results and the Deit family …
-
Hi,
i want to use cosub trainging method for custom images classification work, but i don't find implemention in the code. Does this work contains the details of cosub training method. If yes, my com…
-
Cannot find it in the paper? Is it the same as training on imagenet?
-
With regards to this sentence:
> In Table 5, we show that the Top-1 accuracy on the validation set of ImageNet-1k improves by more than +2% when the backbone is fine-tuned.
What was the learning…
-
- https://arxiv.org/abs/2104.12533
- 2021
昨年は、Transformerモジュールを視覚問題に適用することが急速に発展しました。
一部の研究者は、Transformerベースのモデルがデータをフィットさせる能力に優れていることを実証しているが、特に学習データが限られている場合には、これらのモデルが過剰にフィットしてしまうことを示す証拠がまだ増えて…
e4exp updated
3 years ago
-
Hi, @lucidrains !
There was a promising research published this month (vs. RoPE-mixed (#25) in March), the so-called LieRE positional encodings generalize the kv-vector rotation to any numbers of d…
-
Hi and thanks a lot for this codebase!
I'm trying to reproduce your results with a Deit-small, but training is very slow, with more than 2h/epoch.
I have tried the deit commands from the readme,…
-
From DeiT III: Revenge of the ViT (https://arxiv.org/pdf/2204.07118.pdf), how long exactly (exact number of hours) does it take to pretrain for 90 epochs on ImageNet21k with 8 V100 GPUs.
-
Hi
I'm trying to reproduce DeiT-III's ImageNet-21k results
Could you please share ViT-B weight which is pretrained on 21k but not fine-tuned on 1k?
It will be a great help to check and validate m…
-
Hi Hugo,
I noticed that the suggested command for pretraining DeiT III on IN-21k set --nb_classes to 1000, which is smaller than the number of classes of IN-21k.
Is it a typo?
Best,
Blakey