janetzki / GUIDE

Create semantic domain dictionaries for low-resource languages

MIT License

4 stars 0 forks source link

Evaluate LRL lemmatization quality #2

Closed janetzki closed 11 months ago

janetzki commented 1 year ago

Goal

As a developer, I want to see how well the lemmatization of LRL works so that I can explain and improve the lemmatization approach.

Motivation: improve lemma groups -> reduce false positives -> increase precision

Tasks

[ ] Scatter plot word pairs by resource allocation index and Levenshtein distance
- see clusters --> discover appropriate thresholds
- evaluate on English --> show true and false connections in green and red
[ ] use English lemmas from WordNet as ground truth