-
# 🚀 Feature request
Currently, training the models from scratch like Roberta do not support whole word masking (e.g., language modeling examples). Only pre-trained models are available. Is it possi…
-
您好,想请问下,你们的BERT-wwm-ext模型及RoBERTa-wwm-ext模型预训练达到收敛的时候,预训练目标mlm及nsp的loss和acc大概是什么水平?
-
我使用的是https://github.com/ymcui/Chinese-BERT-wwm#%E4%B8%AD%E6%96%87%E6%A8%A1%E5%9E%8B%E4%B8%8B%E8%BD%BD
上的BERT-wwm, Chinese 放在了 pretrained文件夹下,不知道如何启用,使用LM模型启动时,加载不到其必须的文件。
ly886 updated
4 years ago
-
bert4keras 0.7.6
tf 2.0.0
keras 2.3.1
请教一个问题.
在data_utils.py中是通过some_texts来将corpus转换成预训练的格式.
我看逻辑是将文档分为句子,然后组成list.文档之间没有区分的标识.
但是bert 预训练中,文章和文章,是有空行. 然后mask 和 predict 是在单个文档之间进行的,而不是跨文档.…
-
无论是在huggingface.co/models上下载了模型加载还是直接用模型名hfl/chinese-roberta-wwm-ext加载,无论是用RobertaTokenizer还是BertTokenizer都会报如下错误:
Traceback (most recent call last):
File "BERTbaseline_pytorch.py", line 727, in
…
-
## Environment info
- `transformers` version: 4.1.0.dev0
- Platform: Linux-4.4.0-139-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow …
-
如题,如果分词,是使用哈工大的ltp进行分词么?
如果使用字级别的可以么?
-
Hi, I am trying to replicate the results by following README
Does `## BERT-large-wwm-uncased` and `## BERT-base-uncased` mentioned in evaluation results same to `rsvp-ai/bertserini-bert-large-squad…
-
# ❓ Questions & Help
I try to using own dataset on tpu with running run_language_model.py, the command is I use below:
python examples/xla_spawn.py --num_cores 8 examples/language-modeling/run_la…
-
你好,请问一下中文预训练模型的下载链接失效了,谷歌和讯飞云的都下载不了,可以麻烦分享一下百度云的么