search

sihaoevery / lambda_vit

Other
18 stars 3 forks source link

Confusion over Algorithm 1 #2

Open negan7879 opened 2 months ago

negan7879 commented 2 months ago

How is the G in line 4 of Algorithm 1 to be interpreted? Where is the definition?

sihaoevery commented 2 months ago

Hi,

G is the task head such as the classification head in Eq. (1). In line 4, we calculate the transfer entropy of the attention layers towards the task head and select the attention layer that has the minimum transfer entropy to the task head.

Best,

Sihao