This PR introduces a question-based accuracy metric, where the overall accuracy for a question is set to 1 if any candidate's score is 1. It also fixes the contextual precision metric to ensure scores do not exceed 1 and adds faithfulness to the generation metrics. Additionally, a batch mechanism is implemented to reduce frequent requests to the engine, which helps prevent errors.
This PR introduces a question-based accuracy metric, where the overall accuracy for a question is set to 1 if any candidate's score is 1. It also fixes the contextual precision metric to ensure scores do not exceed 1 and adds faithfulness to the generation metrics. Additionally, a batch mechanism is implemented to reduce frequent requests to the engine, which helps prevent errors.
Screenshots