Visualization question

[King4819]

Awesome work !!! I want to ask that is there any specific reason to seperate the "pruned & reused later" and "pruned & reused in the next layer" token ? As shown below:

[kaikai23]

This shows that if a token is to be reused, it is likely it will be reused immediately after it's pruned. This is intuitive, because when a token is not used for consecutive several layers, its feature would be far from what the current layer's features represent and thus becomes unusable.

[King4819]

@kaikai23 ,Thanks for your reply. I also want to ask that in the left figure, how is the ratio of tokens calculated ? Does the three columns sum to the token prune ratio in that layer ? For example, in the fourth layer, the three columns sum to approximately 0.45, so the prune ratio of tokens in the fourth layer is 0.45 ?

[kaikai23]

Hi, the first two columns add up to the pruning ratio of that layer. So fo the 4th layer, the pruning ratio is about 0.3.

The third column is a subset of the second column. Note that for layer 11, since all later layers = the next layer, column 2 equals column 3.

[King4819]

@kaikai23 Thanks for your reply