-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
When evaluating a RAG retrieval service using the llama-index evaluation method, I encou…
-
We might want to look into some new RAG chunk retrieval methods and evaluation metrics. This [short tutorial](https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/) covers some i…
-
can you please guide me on this,
I have two accounts: [arundhathi@one.ai](mailto:arundhathi@one.ai) and [arundhathi@two.ai](mailto:arundhathi@two.ai). On Azure, I have a subscription under the one.…
-
Thank you for your great work!
I wonder if it can be integrated into popular evaluation frameworks like lmms_eval or vlmevalkit for easier use by everyone?
-
Determine the best chunk size for our application.
The application should be able to keep track of individuals, so that RAG can make an appropriate prompt to feed to LLM, and the application can gi…
-
## 🐛 Bug
Hi TorchMetrics Team,
In the following example, nDCG calculation using GPU tensors spent 2 times longer the time using CPU tensors and numpy array.
### To Reproduce
The codes were…
-
Hey, Congratulations for your perfect and creative work.
when I read the implementation code here, I am very confused about [SampledSoftmaxLoss](https://github.com/facebookresearch/generative-recomme…
-
I observed that some datasets such as **CmedqaRetrieval, CMedQAv1, CMedQAv2** Built from QA datasets and converted to 'query-pos-neg' format. Do you have 1 instruction for building this data?
QA data…
-
I've encountered issues while trying to reproduce the BM25 results mentioned in the documentation. I've faced the challenges:
How does the script handle files with more context than the tokenizer c…
-
> Please provide us with the following information:
> ---------------------------------------------------------------
### This issue is for a: (mark with an `x`)
```
- [ ] bug report -> please…