Open gordon0414 opened 1 week ago
Hello, I was wondering if we can replace the BERT for current supported models
Languages.JP: BASE_DIR / "bert" / "deberta-v2-large-japanese-char-wwm", Languages.EN: BASE_DIR / "bert" / "deberta-v3-large", Languages.ZH: BASE_DIR / "bert" / "chinese-roberta-wwm-ext-large",
Since the current models are mostly large variants of the models, but I think smaller BERT models will still be capable for our use case.
large
I mean by smaller variants such as
I suppose these variants won't cause any issues since the tokenizers are identical to the original ones
My hypothesis is that if we fine-tune our model with the new BERT models, it will seamlessly incorporate with the current system.
I haven't checked, but maybe as you guess we can, and you can try it.
okay, maybe I should try it this weekend!
Hello, I was wondering if we can replace the BERT for current supported models
Since the current models are mostly
large
variants of the models, but I think smaller BERT models will still be capable for our use case.I mean by smaller variants such as
I suppose these variants won't cause any issues since the tokenizers are identical to the original ones
My hypothesis is that if we fine-tune our model with the new BERT models, it will seamlessly incorporate with the current system.