-
In `deberta.mlm`, `MaskedLayerNorm ` is not imported from `deberta.ops`, and `PreLayerNorm` is undefined.
And I'm not sure if `deberta.mlm` contains codes for pretraining?
-
Does bert-score support DeBERTaV3?
https://huggingface.co/microsoft/deberta-v3-large
If not, what types of work would this require?
-
Hi,
Is there any tutorial that we can refer to so that we could serve a deberta model using fastertransformer in Triton?
I think the steps would be:
1. Convert a deberta-v2 model into fastertrans…
-
Since there is not code example provided for pytorch, I wrote my own version below. But it turned out it gave different result than using the huggingface model. Anything wrong with my code?
```
impo…
-
Hi @DaoTranbk and @HyTruongSon,
many thanks for open sourcing the repo for ViDeBERTa!
I'm very interested in the v3 pretraining of a DeBERTa model. In the current version of the pretraining code…
-
### System Info
Python version: 3.8.15
Transformers version: 4.24.0
### Who can help?
@ArthurZucker, @younesbelkada
### Information
- [ ] The official example scripts
- [X] My own mod…
-
Hi, in appendix 4 of the paper you describe results on NLG tasks and a DeBERTa-MT base model pretrained on wikitext-103, using unilm task formulations.
Are you going to release this model and the c…
-
I've setup the environment according to the instruction, and
try pretraining the rtd model as follows:
_Deberta/experiments/language_model/bash rtd.sh deberta-v3-base_
But I got error message …
-
Provide a space for users to request help and support related to using the DeBERTa model.
-
In the current version, when running:
```python
from DeBERTa import deberta
vocab_path, vocab_type = deberta.load_vocab(pretrained_id='base')
tokenizer = deberta.tokenizers[vocab_type](vocab_pat…