Adding 3 metrics

[PierreColombo]

Hello ! I hope you are doing great !

Could you add these 3 metrics:

• BaryScore (oral EMNL 2021)
• DepthScore
• InfoLM (best student paper award AAAI 2022)

They are available here: https://github.com/PierreColombo/nlg_eval_via_simi_measures on a unified format !

Cheers,

[lvwerra]

Hi @PierreColombo

These metrics look interesting and I think they would be great community metrics. You could add them with the CLI:

evaluate-cli create "Bary Score"

This will create a folder with the necessary skeleton fir the metric and you will also see instructions on how to fill the gaps. That way they will be shared under your name. Would you like to give it a shot? Happy to help should you encounter any difficulties. We also have a guide in the documentation: https://huggingface.co/docs/evaluate/creating_and_sharing

[PierreColombo]

If you want to do it this would be great! You can take credit for it ! Thanks :)

On Wed, Jun 8, 2022, 2:53 PM Leandro von Werra @.***> wrote:

Hi @PierreColombo https://github.com/PierreColombo

These metrics look interesting and I think they would be great community metrics. You could add them with the CLI:

evaluate-cli create "Bary Score"

This will create a folder with the necessary skeleton fir the metric and you will also see instructions on how to fill the gaps. That way they will be shared under your name. Would you like to give it a shot? Happy to help should you encounter any difficulties. We also have a guide in the documentation: https://huggingface.co/docs/evaluate/creating_and_sharing

— Reply to this email directly, view it on GitHub https://github.com/huggingface/evaluate/issues/127#issuecomment-1149876910, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLTNJ5S5CKHBHEEMO7PIQ3VOCJVXANCNFSM5YARWVFQ . You are receiving this because you were mentioned.Message ID: @.***>

[lvwerra]

Hi @PierreColombo, I added a skeleton for Bary Score here: https://huggingface.co/spaces/lvwerra/bary_score/tree/main

From the template only the bary_score.py needs to be adapted and I copied your class to score.py. What's missing is filling the documentation in bary_score.py and writing the README.

It would be great if you could take over and finish the documentation as you are the author of the metric and you could use the same approach for the other metrics.

[cakiki]

Happy to work on this if @PierreColombo does not have time!

[PierreColombo]

Thanks @cakiki it would be terrific ! All the 3 metrics have the same signature and structure ! Many thanks :)