-
代码可以在qnli、QQP、sst2任务上正确复现,但是在mnli任务上结果很差劲,第一步layers_distill的准确率为43.8,第二步一直是10%左右,不清楚别的同学有没有遇到该问题?
-
## 一言でいうと
コンピューティングリソースが少ない場合、モデルとしてはELECTRAがよいとした記事。DistilBERT/TinyBERTより少ないパラメーターで、 GLUEベンチマークで同等精度を記録できる。
### 論文リンク
https://arxiv.org/abs/2104.02756
### 著者/所属機関
François Mercier
*…
-
trying to run nboost with Tensorflow using Biobert model. Getting the following stack trace.
```
:resolve_model:[__i:res: 43]:Extracting "/usr/local/lib/python3.6/dist-packages/nboost/.cache/biobert…
-
Hi, huawei-noah team.
Thank you for sharing the code of your interesting work, TinyBERT.
I wonder which factors resulted in the performance improvement on the RTE and SQuAD 2.0 datasets, comparin…
-
-
Thank you for releasing and maintaining this repo.
Can you please provide a link to the wikipedia dataset, and additional datasets required to train TinyBert from scratch, in the required format (tex…
-
According to your [code](https://github.com/huawei-noah/Pretrained-Language-Model/blob/a8a705e9c8c952e078b45d1091d3f0ed161483d8/TinyBERT/task_distill.py#L1069) in `task_distill.py`, the evaluation on …
-
According to https://sbert.net/examples/applications/retrieve_rerank/README.html#re-ranker-cross-encoder
the cross encoder "outputs a single score between 0 and 1". I do get these results with some u…
niebb updated
3 years ago
-
1. X2bolt -d onnx -m model -i PTQ #输出为model_ptq_input.bolt
2. ./post_training_quantization -p model_ptq_input.bolt -i INT8_FP32 -b true -q NOQUANT -c 0 -o false
3. 推理报错如下:
[ERROR] thread 121948 fil…
-
您好!
TinyBERT预训练模型参数有`fit_denses.0.weight`一直到`fit_denses.6.bias`,但是modeling.py组网`TinyBertForSequenceClassification`中只有一个共享参数是`fit_dense`。
在这里想请教一下,是建议忽略模型参数,还是建议更改modeling.py,哪种使用方式是建议的呢?
感谢:)