Add support for the XLM model from huggingface/pytorch-transformers

nikitakit / self-attentive-parser

High-accuracy NLP parser with models for 11 languages.

https://parser.kitaev.io/

MIT License

871 stars 153 forks source link

Add support for the XLM model from huggingface/pytorch-transformers #52

Closed freesunshine0316 closed 3 years ago

freesunshine0316 commented 4 years ago

Dear all,

I added additional code to support the facebook XLM model. After initial experiments, the model with XLM shows comparable performances with BERT-Chinese on CTB. I guess XLM can be better than BERT after a comprehensive hyperparameter search.

nikitakit commented 3 years ago

Sorry that I haven't been monitoring this GitHub repo in a long long time. The transformers library has come a long ways since I released benepar (the first benepar release actually came before BERT was even published). As of v0.2.0a0, benepar can use almost any model provided by huggingface with a simple hyperparameter change. I'm closing this PR to reflect that.