search

phycholosogy / RAG-privacy

The code for paper "The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)", exploring the privacy risk on RAG.
26 stars 3 forks source link

Evaluate #3

Open redwyd opened 1 month ago

redwyd commented 1 month ago

When I evaluated the generated results after running the third step, the following error occurred： Traceback (most recent call last): File "/RAG-privacy/evaluation_results.py", line 767, in eval_results(settings, title_table, table_list) File "/RAG-privacy/evaluation_results.py", line 603, in evalresults sources, outputs, contexts = getdata(path, model, tem, p, seq, gen) File "/RAG-privacy/evaluation_results.py", line 120, in get_data k = len(sources) // len(outputs) ZeroDivisionError: integer division or modulo by zero

The error occurs in this code. Is there something wrong?: outputs = output.split('\n===================\n')[:-1]

cryingInitial commented 4 weeks ago

Evaluation not works here too I tried to solve the problems. But evaluation is virtually impossible

Ahram-41 commented 1 week ago

It seems because the developer didn't update the evaluation.py file. In the commit history of generate.py, information is split by '\n===================\n in txt files, so when running evaluation.py, the splitter cannot split the str into list by '\n===================\n' and zero division error occurs