Open negan7879 opened 2 months ago
Hi,
G is the task head such as the classification head in Eq. (1). In line 4, we calculate the transfer entropy of the attention layers towards the task head and select the attention layer that has the minimum transfer entropy to the task head.
Best,
Sihao
How is the G in line 4 of Algorithm 1 to be interpreted? Where is the definition?