Closed sadanyh closed 3 years ago
Hi, right now the XLM-R-related code is not included in this repo as we found that it underperformed mBERT as in the paper. I'll upload the XLM-R code to a different branch.
Hi, thanks for the awesome tool. Any update on using XLM-R Code? In which branch can I find it?
Hi @sujoysarkarcs, sorry for the delay. I'll upload the code by 08/16.
Uploaded the code to https://github.com/neulab/awesome-align/tree/xlmr.
Thank you.
Hi Thank you awesome-align team for the nice tool and the demo. I am trying to use XML-R with your model instead of M-BERT. I get an error with this line: out_src = model(ids_src.unsqueeze(0), output_hidden_states=True)[2][align_layer][0, 1:-1]
My guess is that the hidden_states and model output for XML-R is different. I think the index for the word embeddings in XML-R is index [-1]. Could you please give your direction on how to solve this, if it is possible?
Thanks