search

nyu-mll / jiant

jiant is an nlp toolkit
https://jiant.info
MIT License
1.64k stars 297 forks source link

DeBERTa support #1293

Closed nlpist closed 3 years ago

nlpist commented 3 years ago

It seems like the newest transformers AutoModel supports DeBERTa (you can find it on AutoModel page https://huggingface.co/transformers/model_doc/auto.html#transformers.AutoModel)

The transformers version that is required by jiant (transformers==3.1.0) doesn't support DeBERTa

So I installed the newest transformers 4.4.0 and jiant doesn't support it and fails with the following error message

  File "/jiant/jiant/tasks/lib/templates/squad_style/core.py", line 8, in <module>
    from transformers.tokenization_bert import whitespace_tokenize                                                                                                                                                                     
ModuleNotFoundError: No module named 'transformers.tokenization_bert'

I wonder what's the easiest way to integrate the DeBERTa support?

Thank you!

jeswan commented 3 years ago

I am currently working on adding DeBERTa here: https://github.com/nyu-mll/jiant/commit/12a34b356c16f08dcbc4d5ff81dc59de17c23624 (this depends on a large refactor to make it easy to add a model - branch: js/feature/easy_add_model). Note this branch is a work in progress.

nlpist commented 3 years ago

@jeswan Thank you for the fast response!