请问作者有做中文数据增强吗？

[hahlw]

Tinybert(4L，"hidden_size": 312,"intermediate_size": 1200,)模型随机初始化，结果相差bert-base 9个点（acc83%降到74%）。这种结果正常吗？

[airaria]

什么任务，什么配置？

[hahlw]

什么任务，什么配置？

文本分类任务，单卡TITAN XP(12g) 训练了30个epoch,选用比较好的epoch测试是75.94%，bertbase是80.02%（之前的指标观测错了）这个指标算是正常吗？我加了attentionloss 但是loss是先上升后下降。这个正常吗？

[airaria]

看起来正常，如果没调过超参，再调一调应该还有上升空间

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.