hotpot.json data is repeated

OSU-NLP-Group / HippoRAG

[NeurIPS'24] HippoRAG is a novel RAG framework inspired by human long-term memory that enables LLMs to continuously integrate knowledge across external documents. RAG + Knowledge Graphs + Personalized PageRank.

https://arxiv.org/abs/2405.14831

MIT License

1.41k stars 117 forks source link

hotpot.json data is repeated #60

Open fantastic-413 opened 1 week ago

fantastic-413 commented 1 week ago

There are two identical data (the 6th and 506th data) in the hotpot.json file. So when you run the results of ircot_hipporag.py on the hotpotqa data, if you interrupt halfway (idx>=6) and rerun the code, due to this mechanism, you will skip the 506th data and only get the results of 999 data. But the results of using 1000 data but with the same data may not necessarily be better than the results of 999 data.

yhshu commented 1 week ago

Thank you for your feedback. We will fix this dataset problem.