microsoft / DynamicHead

MIT License
624 stars 60 forks source link

do all channel features share same (alpha_1, beta_1, alpha_2, beta_2)? #11

Open PatrickRuan opened 3 years ago

PatrickRuan commented 3 years ago

Hi, Sir, At equation 5, it seems we only have a set of (alpha_1, beta_1, alpha_2, beta_2). Dose it tell us all channel features slide work with same activation?

Is it possible to set (alpha_1_c1, beta_1_c1, alpha_2_c1, beta_2_c1) for channel 1, (alpha_1_c2, beta_1_c2, alpha_2_c2, beta_2_c2) for channel 2... refer to dynamic relu fig. 2-b?

looking forward to having your answer soon.

Thanks, Patrick