List all potential test benchmarks

[faneshion]

List all most used datasets in RAG researches, and we will add them to the benchmarks.

• [ ] THUDM/webglm-qa from huggingface: https://huggingface.co/datasets/THUDM/webglm-qa
• [ ] NaturalQuestions from huggingface: https://huggingface.co/datasets/natural_questions
• [ ] #64
• [ ] Trivia QA from huggingface: https://huggingface.co/datasets/trivia_qa
• [ ] Hotpot QA from huggingface: https://huggingface.co/datasets/hotpot_qa
• [ ] WikiEval from huggingface: https://huggingface.co/datasets/explodinggradients/WikiEval

[FBzzh]

• [ ] MMLU from huggingface: https://huggingface.co/datasets/cais/mmlu
• [ ] PopQA from huggingface: https://huggingface.co/datasets/akariasai/PopQA
• [ ] WebQuestions from hugginggace: https://huggingface.co/datasets/web_questions
• [ ] FEVER from hugginggace: https://huggingface.co/datasets/fever
• [ ] FeTaQA from hugginggace: https://huggingface.co/datasets/DongfuTingle/FeTaQA

[FBzzh]

• [ ] MedMCQA from hugginggace: https://huggingface.co/datasets/medmcqa
• [ ] GSM8K from hugginggace: https://huggingface.co/datasets/gsm8k
• [ ] BBH from github: https://github.com/suzgunmirac/BIG-Bench-Hard
• [ ] SQuAD from hugginggace: https://huggingface.co/datasets/squad
• [ ] SQuAD_v2 from hugginggace: https://huggingface.co/datasets/squad_v2
• [ ] Wizard-of-Wikipedia(WoW) from hugginggace: https://huggingface.co/datasets/chujiezheng/wizard_of_wikipedia

[Wenshansilvia]

Select and implement typical benchmarks, collect RAG papers that utilized these benchmarks, and try to reproduce evaluation result in the paper.

1. List benchmark and related papers & metrics.
2. Produce testset using baseline RAG in the paper. Pack testset as dataset format and upload to HuggingFace.
3. Reproduce evaluation result in RAGEval.

Eli5 @QianHaosheng , ASQA @bugtig6351 , Fever @henan991201