-
**Objective:**
Create an advanced Embedding Evaluator tool designed to assess the quality of embeddings generated for context injection into a Language Large Model (LLM). The tool will use specific m…
-
- [ ] Metadata extraction - HM
- [ ] Answering introductory questions - GK
- [x] Answering Open Chat questions - moved to genai-apps/aggrag#36
-
**Describe the feature**
Typically OpenAI API's are protected with some API_KEY, so to run requests to such API's it is necessary to specify API_KEY in a call.
Is it planned to add such functionalit…
vlsav updated
2 months ago
-
**Describe the bug**
Following the instructions in https://github.com/microsoft/Olive/tree/main/examples/whisper, I carried out the following steps for model optimization:
1)
```
python3 prepare…
-
I have a llama-2-7b.gguf quantized in Q4_K_M that I run with llama.cpp server. The inference works fine.
This is the code (found from [here](https://github.com/EleutherAI/lm-evaluation-harness/iss…
-
**Describe the bug**
1. Summarization scores change depending on:
- the type of assessment questions being asked
- the number of assessment questions provided
- whether a developer provides the …
-
### Bug Description
I am trying to use one of the llama_datasets found at `https://llamahub.ai/l/llama_datasets/Evaluating%20LLM%20Survey%20Paper%20Dataset?from=llama_datasets`.
I tried running t…
-
[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I have a code using the package ragas that works in a noteboo…
-
### Bug Description
vector_store_address=
vector_store_password=
search_client = SearchClient(
endpoint=vector_store_address,
index_name=INDEX_NAME,
credential=AzureKeyCredential…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hi, I have a `LabelledRagDataset` created with `RagDatasetGenerator`. Now how can …