Question
I am just asking if Ragas is still working with Tonic Validate since the code provided in the documentation is not working: https://validate.tonic.ai/
Below is a code that is supposed to calculate the score on a benchmark composed of one question.
Code Examples
from ragas import evaluate
import json
from datasets import Dataset
from tonic_ragas_logger import RagasValidateApi
#Case of no restriction
data_samples = {"question": [], "answer": [], "contexts": [], "ground_truth": []}
with open("reponse_rag_testset_one_question.json", "r", encoding="utf-8") as file:
reponse_data = json.load(file)
for item in reponse_data:
data_samples["answer"].append(item["answer"][0])
data_samples["question"].append(item["question"])
data_samples["contexts"].append(item["context"])
data_samples["ground_truth"].append(item["ground_truth"])
dataset = Dataset.from_dict(data_samples)
scores = evaluate(
dataset, metrics=metrics, llm=azure_model, embeddings=azure_embeddings
)
#send the score to Tonic Validate
validate_api = RagasValidateApi("G6ph3yz5hk47bQIUtRKMqfF7y8GgiJ-uBLrv6-GNmtU")
validate_api.upload_results("f5aa069e-924b-4723-a5b5-ced06f9b8ab7", scores)
Error
I also provide here the error message when running the code:
Traceback (most recent call last):
File "c:\Users\adle.bensalem\Downloads\RAG-HES-SO\ragas_evaluate.py", line 90, in <module>
validate_api.upload_results("f5aa069e-924b-4723-a5b5-ced06f9b8ab7", scores)
File "C:\Users\adle.bensalem\AppData\Local\Programs\Python\Python312\Lib\site-packages\tonic_ragas_logger\ragas_validate_api.py", line 56, in upload_results
run = self.__convert_to_run(results)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\adle.bensalem\AppData\Local\Programs\Python\Python312\Lib\site-packages\tonic_ragas_logger\ragas_validate_api.py", line 124, in __convert_to_run
return Run(
^^^^
File "C:\Users\adle.bensalem\AppData\Local\Programs\Python\Python312\Lib\site-packages\pydantic\_internal\_dataclasses.py", line 141, in __init__
s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Run
llm_evaluator
Field required [type=missing, input_value=ArgsKwargs((), {'overall_...onner.'])], 'id': None}), input_type=ArgsKwargs]
Question I am just asking if Ragas is still working with Tonic Validate since the code provided in the documentation is not working: https://validate.tonic.ai/
Below is a code that is supposed to calculate the score on a benchmark composed of one question.
Code Examples
Error I also provide here the error message when running the code: