ybracke / transnormer

A lexical normalizer for historical spelling variants using a transformer architecture.
GNU General Public License v3.0
6 stars 1 forks source link

Evaluation: Implement evaluation #61

Closed ybracke closed 10 months ago

ybracke commented 1 year ago
ybracke commented 10 months ago

Sketch for evaluation process

1. Generating predictions

2. Compute evaluation metrics

Evaluation returns a JSONL file with metrics. Example output:

{"n": 100,  "acc_harmonized": 0.95,  "dist_normalized": 0.1} 

:white_check_mark: Done