search

huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
https://huggingface.co/docs/evaluate
Apache License 2.0
2.04k stars 260 forks source link

Add Average Normalized Levenshtein Similarity (ANLS) #321

Open NielsRogge opened 2 years ago

NielsRogge commented 2 years ago

Document visual question answering (DocVQA) models, like LayoutLM and Donut, are evaluated using the Average Normalized Levenshtein Similarity metric (ANLS for short).

Would be very useful for the community to have this metric in the evaluate library.

The metric was introduced here for DocVQA: https://rrc.cvc.uab.es/?ch=17&com=tasks.

image (4)

medahmedkrichen commented 1 year ago

412

I tried to add the Average Normalized Levenshtein Similarity (ANLS), but this is my first contribution to open source so please tell me if there is anything I can correct. Thanks!

QuadV commented 1 year ago

I've corrected the Flake8 errors. Please let me know if I should look into anything else for this issue.

NielsRogge commented 1 year ago

Cc @lvwerra would it be possible to review this?

Feel free to assign someone else in case you don't have the bandwidth