-
When you tested the ATS and EvoViT pruning methods, how did you exactly incorporate the CLS token?
As you mention the CLS token is not "natural" for dense tasks, but given you use a DeiT backbone, yo…
-
Hello,
Thank you for providing this excellent codebase! It has been very helpful for my work. I was wondering if it would be possible to extend the current implementation to support the DeiT model,…
-
Thank you for sharing your excellent work.
I'm interested in applying it to my research, so I followed your instructions to reproduce the results. However, when training on ImageNet, I repeatedly e…
-
I find your paper on Vision Mamba very interesting. However, when using your code, I encountered a problem (which may well be normal behavior). When analyzing GPU memory consumption and FPS for Vim ve…
-
- [ ] Confirm structure and web pages by Tuesday, May 18
- [ ] Write content for the pages by Tuesday, June 1
-
Hello,
In your paper, it was mentioned that you utilized an INT8 quantized Deit model. I noticed that there is a reformat unit in each HCE for type conversion between int8 and float32, but I did no…
-
### 🚀 The feature
As the title suggest, add the data augmentation from https://arxiv.org/abs/2204.07118
### Motivation, pitch
This seems to be a simple recipe with good results and the Deit family …
-
Hi, I'm curious about if Theia can change the backbone of the student model. Now you are using DEIT-base-patch-16-224, I want to use a pre-trained larger model like clip-vit-base-patch32 to distill ot…
-
## Reference
- 2020-12 **[DeiT]** Training data-efficient image transformers & distillation through attention [[Paper](https://arxiv.org/abs/2012.12877)] [[Code](https://github.com/facebookresearch/d…
-
privacy_engine = PrivacyEngine(
File "/home/idris/.local/lib/python3.10/site-packages/private_transformers/privacy_engine.py", line 176, in __init__
raise ValueError(
ValueError: Model type …