search

stanford-futuredata / ARES

Automated Evaluation of RAG Systems
https://ares-ai.vercel.app/
Apache License 2.0
452 stars 49 forks source link

strong negative generation #1

Closed SimonYx42 closed 9 months ago

SimonYx42 commented 10 months ago

in the paper, there is a strong negative generation method to construct negative samples for LLM judges training, but in the repo, I can't find any code about this, is it actually not used to produce the final result in the paper?

jonsaadfalcon commented 9 months ago

Hello, thank you for your interest in ARES! In _Generate_Synthetic_Queries_andAnswers.py, we set the strong negatives collected with the _lower_bound_fornegatives parameter.

We manually set it to 20 but you can change it to a lower value if you want stronger negatives or a higher value if you want weaker negatives.