search

grammarly / gector

Official implementation of the papers "GECToR – Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)
Apache License 2.0
900 stars 214 forks source link

Achievement of metric values on conll-2014 #124

Closed kristinaivanova97 closed 3 years ago

kristinaivanova97 commented 3 years ago

Hello! I am trying to obtain the same metrics on conll-14 test dataset as mentioned in the GECToR paper and with the instructions on the GitHub page I was able to obtain only these:

Precision : 0.4167 Recall : 0.3538 F_0.5 : 0.4024

Can you help me with obtaining better values (from article)? I've used xlnet_0_gector.th weights and vocabulary from the GitHub. Other params:

max_len=128, min_len=0, min_error_probability=0, special_tokens_fix=0

skurzhanskyi commented 3 years ago

Hi! Here's the table with min_error_probability & additional_confidence you need to use: https://github.com/grammarly/gector#pretrained-models