search

KLUE-benchmark / KLUE

📖 Korean NLU Benchmark
https://klue-benchmark.com
Creative Commons Attribution Share Alike 4.0 International
554 stars 55 forks source link

NLI performance #19

Closed LittlePea13 closed 3 years ago

LittlePea13 commented 3 years ago

Hi there, Could you share the hyperparameters for the NLI task using Roberta-large? I am unable to replicate the 0.89 performance, and after trying the different setups I get a performance around 0.86 tops. Also, is there any plan to release checkpoints for each of the tasks?

Thanks!

inmoonlight commented 3 years ago

Hi, @LittlePea13 Thanks for your interest on KLUE!

KLUE team is planning to release finetuning pipeline and training script at the end of July. You would get detailed training hyperparameters after the release. About releasing checkpoints, frankly speaking, we haven't think of it, since our results are "baselines" - we didn't put a lot of effort to get maximum finetuning results. But I will think about it! Thx