search

neulab / awesome-align

A neural word aligner based on multilingual BERT
https://arxiv.org/abs/2101.08231
BSD 3-Clause "New" or "Revised" License
321 stars 46 forks source link

A bug(maybe) #55

Open shaoyangxu opened 1 year ago

shaoyangxu commented 1 year ago

Hi, dear ziyi~ I found in your code, the bert output weights are not set to be the same as the input embedding, which can be proved in here(In detail, the code didnt set the weight of BertLMPredictionHead.decoder to be the same as the weight of BertModel.embeddings). I think maybe it is you who deliberately modified the source code of bert LM into this way. Why? Will it influence the final result?

zdou0830 commented 1 year ago

Hi, yes the input and output embedding layers are not shared because I didn't define get_input/output_embeddings in BERT. This was not intentional but I found this has almost no impact on the alignment performance.