How can I use the HHEM model to evaluate my LLM after finetuning?

vectara / hallucination-leaderboard

Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

https://vectara.com

Apache License 2.0

1.16k stars 45 forks source link

How can I use the HHEM model to evaluate my LLM after finetuning? #50

Open zjq0455 opened 1 month ago

zjq0455 commented 1 month ago

Thank you for your contributions!

I was wondering whether it is possible or how can I use the HHEM model to evaluate a LLM after finetuning using our specific dataset?

forrestbao commented 1 month ago

hi @zjq0455 thanks for your interest in HHEM. Yes, of course. Just note that HHEM's context window is 512 tokens. We are working on a more powerful open source version to remove this restrictions.