What are the parameters that controls the lower bound of pruned number of heads, pruned weight (s) and pruned size of head (r)?

VITA-Group / UVC

[ICLR 2022] "Unified Vision Transformer Compression" by Shixing Yu*, Tianlong Chen*, Jiayi Shen, Huan Yuan, Jianchao Tan, Sen Yang, Ji Liu, Zhangyang Wang

MIT License

45 stars 3 forks source link

What are the parameters that controls the lower bound of pruned number of heads, pruned weight (s) and pruned size of head (r)? #6

Open xjaweiofj opened 9 months ago

xjaweiofj commented 9 months ago

I only found r_ub, s_ub, which stands for the upper bound of r and s. However, I cannot find variable or parameters that is related to the lower bound of r and s.