search

TalSchuster / CrossLingualContextualEmb

Cross-Lingual Alignment of Contextual Word Embeddings
MIT License
98 stars 9 forks source link

What layer is aligned and what the English alignment matrix for? #6

Closed OfirArviv closed 5 years ago

OfirArviv commented 5 years ago

Hi,

I've read your article and the readme here and I'm a little confused about 2 things:

  1. You state

    We provide the alignment of the first LSTM output of ELMo to English

and also:

Note that our alignments were done on the second layer of ELMo

My understanding is that "fist LSTM output" is not the second layer. Am I misunderstanding the quote? What layer do you align?

  1. If the alignment matrices aligned the models to the English model, what is the purpose of the English alignment matrix?

Thanks!

TalSchuster commented 5 years ago

Hi Ofir,

Sorry for the confusion. I've updated the README file now to have alignments for all 3 layers. The links in the table are for the first "contextualized" layer (after one layer of LSTM).

The English files simply contain the identity matrix divided by the average norm for that layer.

OfirArviv commented 5 years ago

Thank you for the clarification!