search

stanford-futuredata / ARES

Automated Evaluation of RAG Systems
https://ares-ai.vercel.app/
Apache License 2.0
486 stars 53 forks source link

Solve llm judge training error #71

Open WJ44 opened 2 months ago

WJ44 commented 2 months ago

Do not filer rows without generated answers when training Context Relevance LLM Judge. Not only does this cause only positive samples to be present in the training set, it also causes a (CUDA) error during training because only one label exists.

WJ44 commented 2 months ago

While going through the code to solve the error I was encountering I noticed some additional things that I believe to be bugs so I fixed those as well.