-
确认个问题,就是bert系列预训练的时候,[mask]位置计算self_attention时候应该也需要计入吧,不计入感觉不太合理,这样80%[MASK],10%不变,10%随机替换token,就没啥用了吧,因为这样[mask]的padding的就没去别的,都访问不到
-
when build for segmentation task,occur the KeyError: 'BEiT is not in the backbone registry'
-
**Describe**
Model I am using UniLM:
I am trying to fine-tune BEiT base patch 16 224 model from pretained beit_base_patch16_224_pt.pth. However I getting the following error:
```
Traceback (most r…
-
I'm training an InfoXLM model on a custom dataset.
To do this, I'm first training a sentencepiece tokenizer and then converting this tokenizer to huggingface format to make the tokenizer compatibl…
-
在lcqmc数据集上微调simbert,在测试集上spearman指标下降一个点,怎么微调simbert呢?
-
Epoch / Total epochs: 41 / 200
100%|██████████| 568/568 [11:36
-
你好!
关于文本摘要生成的任务
我的数据集是train.src和train.tgt两个文件分别装有text和summarization请问应该如何封装dataset
-
**Describe the bug**
Model I am using (UniLM, MiniLM, LayoutLM ...): probably LayoutXLM, but I don't think it matters.
The problem arises when using:
* [x] the official example scripts: (give det…
-
UniLM team, Awesome work!! I am able to generate very good quality questions & am thoroughly impressed with some particularly generated questions, they are simply amazing, generative models are the fu…
-
ModuleNotFoundError: No module named 'fused_layer_norm_cuda'