search

gomate-community / rageval

Evaluation tools for Retrieval-augmented Generation (RAG) methods.
Apache License 2.0
119 stars 9 forks source link

Add distinct_1/2 metric #108

Closed moshesbeta closed 1 week ago

moshesbeta commented 3 months ago

I add a new evaluation metric, Distinct 1/2, for the generate task evaluation. I have uploaded the new scripts "_answer_distinct12.py" and "test_answer_distinct12.py", and the modified version of "init.py".

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.17%. Comparing base (019ffe3) to head (697f160). Report is 22 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #108 +/- ## ========================================== + Coverage 81.66% 82.17% +0.50% ========================================== Files 33 34 +1 Lines 1129 1161 +32 ========================================== + Hits 922 954 +32 Misses 207 207 ``` | [Flag](https://app.codecov.io/gh/gomate-community/rageval/pull/108/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gomate-community) | Coverage Δ | | |---|---|---| | [](https://app.codecov.io/gh/gomate-community/rageval/pull/108/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gomate-community) | `82.17% <100.00%> (+0.50%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=gomate-community#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.