search

microsoft / DeBERTa

The implementation of DeBERTa
MIT License
1.91k stars 216 forks source link

DeBERTaV3 small & xsmall pre-training configuration? #94

Open zealotjin opened 2 years ago

zealotjin commented 2 years ago

Would it be possible to share configurations used for training the small and xsmall versions of DeBERTaV3? Similar to the files provided in the experiments/language_model/ directory (i.e deberta_base.json).

Thank you in advance 😄

iiLaurens commented 1 year ago

Would also like to see how I could continue pre-training DeBERTa V3 (x)small

stefan-it commented 1 year ago

Hi @zealotjin,

you can find these configuration files on the Hugging Face Model Hub, e.g. deberta-v3-base looks like:

https://huggingface.co/microsoft/deberta-v3-base/blob/main/config.json

Incl. small and xsmall :)