ymcui / Chinese-ELECTRA

Pre-trained Chinese ELECTRA(中文ELECTRA预训练模型)
http://electra.hfl-rc.com
Apache License 2.0
1.4k stars 171 forks source link

一个原理上的疑问 #81

Closed Ricardokevins closed 1 year ago

Ricardokevins commented 1 year ago

请问从框架图来看,ELECTRA是一个Generator+Detector 但是从训练过程里,二者并没有特别显示的联动。 是否可以先单独的训练一个MLM,然后保存这个过程的中间结果 再用大量的中间结果来训练Detector,而不是像论文里的那样联合训练?

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 1 year ago

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.