search

vdobrovolskii / wl-coref

This repository contains the code for EMNLP-2021 paper "Word-Level Coreference Resolution"
MIT License
104 stars 37 forks source link

Better accuracy with XLnet #34

Closed LifeIsStrange closed 2 years ago

LifeIsStrange commented 2 years ago

@vdobrovolskii Hi, I think you would beat the state of the art once again on coreference resolution by switching your Roberta with XLnet. Xlnet consistently outperform Roberta and often significantly.

Thanks for your hard work.

vdobrovolskii commented 2 years ago

Thank you for your suggestion!

LifeIsStrange commented 2 years ago

send me an update if you try that approach :) There is also the topic of next gen optimizers/activation functions (mish, lookahead, gradient centralization, etc https://github.com/lessw2020/Ranger21)

LifeIsStrange commented 2 years ago

For example I recommended XLnet to a researcher that had the SOTA in dependency parsing and it worked, he outperformed his previous result (BERT) and is still to this day the #1 SOTA on pen treebank