-
Hi authors,
Thanks for the great job. I am a little bit confused about eval.py. In the paper, accuracy is used as the evaluation metric for arc_challenge, but in the actual code, match is indeed us…
-
We might want to look into some new RAG chunk retrieval methods and evaluation metrics. This [short tutorial](https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/) covers some i…
-
This issue is used to track all misalignments between python azure search and csharp ones.
link to python-azure-search: https://github.com/Azure-Samples/azure-search-openai-demo
-
Hi, the framework and paper look very promising, but, unfortunately, I've been unable to get any of the tutorials to work. Neither in a Colab notebook nor locally on my mac.
When I try with a Colab…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hi, I have a `LabelledRagDataset` created with `RagDatasetGenerator`. Now how can …
-
Hello, I don't know what I'm doing wrong. I received the following error as indicated in the title.
My input was as shown on this website: :
[Hugging Face - Ger-RAG-eval](https://huggingface.co/da…
-
- **Package Name**: azure-ai-generative
- **Package Version**: 1.0.0b7, 1.0.0b8
- **Operating System**: VS Code Dev Container running in a Mac OSX M1
- **Python Version**: 3.11
**Describe the bu…
-
List all most used datasets in RAG researches, and we will add them to the benchmarks.
- [ ] THUDM/webglm-qa from huggingface: https://huggingface.co/datasets/THUDM/webglm-qa
- [ ] NaturalQuestion…
-
when comparing local llama2 model with GPT4, got error:
ERROR Failed to send a test chat completion to the GPT deployment due to error: …
-
**Describe the bug**
I use ragas to evaluate my project,the dataset comes from the project, when I use 100 number of data,it is normal and could get the evaluate result.Then when I use more than 300 …