Closed King4819 closed 2 months ago
@kaikai23 Thanks for your reply. If I change the arg self.ratio_per_sample to False (default True), it will perform fixed token ratio loss (does not average token usage across images within a batch, thereby penalizing each image towards the same keeping ratio) ?
self.ratio_per_sample=False corresponds to dynamic ratio loss, and Ture correpsondes to fixed ratio loss. I think we used dynamic ratio loss by default? https://github.com/uzh-rpg/svit/blob/d2472c13e667f4ef4c098898faec2e6cbe6c8e01/mmdet_custom/models/backbones/base/selective_vit.py#L100
Oh I get it. Thanks for your reply
Hi, in your work you utilize dynamic pruning ratio loss (averaged across all tokens and across images in a batch, resulting in more tokens allocated for complex images and fewer tokens for simpler images), which implements the below equation:
I would like to ask that how to modify the token ratio loss to fixed (does not average token usage across images within a batch, thereby penalizing each image towards the same keeping ratio) as most pruning methods (e.g., DynamicViT)
Thanks a lot. This is such an awesome work !!!