search

Alibaba-NLP / ACE

[ACL-IJCNLP 2021] Automated Concatenation of Embeddings for Structured Prediction
Other
296 stars 44 forks source link

How many embeds are connected? #50

Closed AliHaiderAhmad001 closed 1 year ago

AliHaiderAhmad001 commented 1 year ago

My greetings, Thank you for these ideas and this work. I would like to know how many embeds are connected (two, three or four?), is it constant during training, and is there an optimal number according to your experience?

wangxinyu0922 commented 1 year ago

Hi, the number of embeddings depends on the language and the task. It usually use more than 10 embedding in ACE training. During training, the embeddings are selected by the controller. You may read the appendix of the paper for more details about the embedding selections.

AliHaiderAhmad001 commented 1 year ago

thank you for replying, I know that, but the question is how many embeddings the controller will choose, is it fixed or task dependent? And if task-dependent, how many embeddings will there be on average (approximately)?

wangxinyu0922 commented 1 year ago

It is task dependent. About half of the candidate embeddings are selected.

AliHaiderAhmad001 commented 1 year ago

Thanks!