THUIR / T2Ranking

T2Ranking: A large-scale Chinese benchmark for passage ranking.
https://huggingface.co/datasets/THUIR/T2Ranking
142 stars 9 forks source link

参考代码,修复了几个数据读取的bug,使用erine-3.0-zh-case不收敛 #4

Closed yyht closed 1 year ago

yyht commented 1 year ago

hi,很棒的工作。 我使用该代码,换成 erine-3.0-zh-base作为初始化,一直不收敛:第10个epoch和第70个epoch的效果差不多

create index done! ##################### MRR @10: 0.06933845813572703 QueriesRanked: 24831 recall@1: 0.005835267211515824 recall@1000: 0.17081189251000573 recall@50: 0.10944909696297044 #####################


new top


**eval, mrr@10: 0.0693384581, report used time:00:33:17, 11

create index done! ##################### MRR @10: 0.07322608771805292 QueriesRanked: 24831 recall@1: 0.0060622876938082266 recall@1000: 0.17147613762486127 recall@50: 0.11481350687787979 #####################


new top


**eval, mrr@10: 0.0732260877, report used time:00:33:46, 71

Deriq-Qian-Dong commented 1 year ago

感谢关注。建议从下面两个角度来调试: 1、换成与本仓库相同的bert-base-uncased-zh看看是否还存在不收敛的问题,不太确定是不是ernie导致的 2、检查数据读取是否正确

Satonio1 commented 1 year ago

hi,很棒的工作。 我使用该代码,换成 erine-3.0-zh-base作为初始化,一直不收敛:第10个epoch和第70个epoch的效果差不多

create index done! ##################### MRR @10: 0.06933845813572703 QueriesRanked: 24831 recall@1: 0.005835267211515824 recall@1000: 0.17081189251000573 recall@50: 0.10944909696297044 #####################

new top

**eval, mrr@10: 0.0693384581, report used time:00:33:17, 11

create index done! ##################### MRR @10: 0.07322608771805292 QueriesRanked: 24831 recall@1: 0.0060622876938082266 recall@1000: 0.17147613762486127 recall@50: 0.11481350687787979 #####################

new top

**eval, mrr@10: 0.0732260877, report used time:00:33:46, 71

请问你找到原因了么,我也遇到了同样的不收敛问题。

Deriq-Qian-Dong commented 1 year ago

已更新代码 @yyht @Satonio1