search

ymcui / Chinese-BERT-wwm

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）
https://ieeexplore.ieee.org/document/9599397
Apache License 2.0
9.56k stars 1.38k forks source link

预训练阶段的loss问题 #222

Closed rtygbwwwerr closed 2 years ago

rtygbwwwerr commented 2 years ago

您好，想请问下，你们的BERT-wwm-ext模型及RoBERTa-wwm-ext模型预训练达到收敛的时候，预训练目标mlm及nsp的loss和acc大概是什么水平？

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 2 years ago

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.