search

HIT-SCIR / ELMoForManyLangs

Pre-trained ELMo Representations for Many Languages
MIT License
1.46k stars 243 forks source link

Task specific combination of the intermediate layer representations #62

Open alberduris opened 5 years ago

alberduris commented 5 years ago

According to [Peters et al., 2018], ELMo is a task specific combination of the intermediate layer representations in the biLM.

The computation of ELMo embeddings is done by a task specific weighting of all biLM layers [1]

image

How is this task thing handled in the ELMoForManyLangs implementation?

To my understanding, this is a general purpose implementation that aims to generate non-task-specific contextual embeddings, so I suppose there are not task-related parameters at all. Am I right?

Thank you very much for shedding some light on this.

Alber.

allanj commented 5 years ago

I think they output the weights for all the layers.