DeBERTa is a very interesting language model, however XLNet is a vastly superior baseline over BERT, XLNet achieves much greater acuracy on key NLP tasks. Unfortunately, for contingent reasons, the scientific community has mostly ignored XLNet and instead chosen to improve BERT variants, this can be considered a historical accident.
It woud be scientifically extremelly interesting for you to port DeBERTa to XLNet and observe that by doing so you just created the best language model for NLP.
DeBERTa is a very interesting language model, however XLNet is a vastly superior baseline over BERT, XLNet achieves much greater acuracy on key NLP tasks. Unfortunately, for contingent reasons, the scientific community has mostly ignored XLNet and instead chosen to improve BERT variants, this can be considered a historical accident. It woud be scientifically extremelly interesting for you to port DeBERTa to XLNet and observe that by doing so you just created the best language model for NLP.
@BigBird01 friendly ping.