search

huggingface / lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
MIT License
832 stars 99 forks source link

Fix BLEURT evaluation errors #316

Closed chuandudx closed 1 month ago

chuandudx commented 2 months ago

These changes address the issues described in: https://github.com/huggingface/lighteval/issues/315

I made the code changes such that it built on the BERTScore changes (https://github.com/huggingface/lighteval/pull/311) that haven't been merged yet, so we see those changes here. Please let me know if there is preference on removing those from this PR. Thank you!

chuandudx commented 2 months ago

Hi @NathanHB :) Just wanted to followup on this PR and welcome any feedback from the concerns we previously discussed. Thank you!

chuandudx commented 1 month ago

Merged the latest changes from main and fixed style error. Thank you!

clefourrier commented 1 month ago

Thanks a lot! FYI, we're working on some other important features atm, but we'll come back to this PR as soon as we can