search

Liyan06 / MiniCheck

MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents [EMNLP 2024]
Apache License 2.0
106 stars 11 forks source link

Adding comparison to leaderboard #5

Open raunak-agarwal opened 3 months ago

raunak-agarwal commented 3 months ago

Hi, I mentioned this on the twitter post. I recently released a preprint which does very similar work where I also used a part of LLM-Aggrefact for my benchmark.

Do you think it's possible to add a comparison to my models to your leaderboard? I can run my models on the updated LLM-Aggrefact benchmark and upload my predictions/results here.

Thanks!

Liyan06 commented 3 months ago

Hi Raunak,

Thanks for your interest!

re: "I can run my models on the updated LLM-Aggrefact benchmark and upload my predictions/results here." To get the most accurate comparison, it would be great that you can have a clean version of the inference code of your model so that I can easily run it on the benchmark myself. Otherwise, it's hard to control whether models are evaluated in the same setting as in the paper.

Let me know if this works for you!