roberta中cls token的计算方法

ymcui / Chinese-BERT-wwm

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

https://ieeexplore.ieee.org/document/9599397

Apache License 2.0

9.66k stars 1.39k forks source link

roberta中cls token的计算方法 #198

Closed wikithink closed 3 years ago

wikithink commented 3 years ago

roberta取消了NSP任务，按道理模型输出应该没有cls的token的向量了，那么cls从哪里计算获取的？计算方法是什么？谢谢

ymcui commented 3 years ago

[CLS]就是第0位的表示啊

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 3 years ago

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.