Open King4819 opened 3 months ago
For the inference stage, we remove (hard prune) the tokens for speedup. https://github.com/PeiyanFlying/SPViT/blob/7ff446dcdbc72fc99097686f7ed6c819e146db3b/vit_l2_3keep_senet_inference.py#L563
We only use gumbel softmax during training. The same approach is used in DynamicViT
@ZLKong Thanks for your reply. If during inference stage it utilizes the technique same as DynamicViT, how do it achieve adaptive pruning? Since the preserve number of tokens is already determined
Hi, I want to ask that what is the strategy at inference stage ? The method utilizes gumbel softmax to generate preserve token mask at training stage, but how to generate preserve token mask at inference stage ?