-
Hi, @lucidrains !
There was a promising research published this month (vs. RoPE-mixed (#25) in March), the so-called LieRE positional encodings generalize the kv-vector rotation to any numbers of d…
-
-
请问有,有没有试过vision transformer类的模型量化?DeiT-S,swin等模型量化精度会有下降。
-
Great work! It is very interesting to handle ViT compression from the perspective of low-frequency components.
I feel a little confused about the attention score (Eq. 8). The definition looks ob…
-
The way I do it in EVA or MMSegmentation is that I do 4 classes (1 background + 3), reduce_zero_label = False and ignore_index=0 in every loss function (CE and Dice in this case), I do those steps and…
-
Thanks for the terrific work and codes!
1. When I run ./benchmark/run_spmm_spatha.sh, I find some shapes are not working. For example, (M=192, K=192, N=3168), which possibly occurs for linear layers …
-
Thanks for the useful repo. I was going through the code, and upon inspection I saw that Vim-T and Vim-S configurations have double the number of blocks (`depth=24`) whereas both Tiny and Small config…
-
Hello, I would like to know how the SwinT and DeiT pie charts that limit memory operations in the EfficientVit paper are analyzed. Can you provide the source code?
-
Hi,
I would like to clarify whether the DeiT-3 checkpoint provided is indeed "pretrained on ImageNet-21K and fine-tuned on ImageNet-1K," as mentioned in the README.
I ran the shape_texture.py sc…
-
Enable support for Diet model using TTNN ops.