-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to use a local LLM in the evaluate function., whe…
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
Error 'openai.APIConnectionError: Connection error.' occurs w…
-
Current candidate
Mistral
https://mistral.ai/
RAG - LlamaIndex
https://github.com/mistralai/cookbook/blob/main/llamaindex_agentic_rag.ipynb
Can be tested out locally and seems to be a good ch…
-
# Proposed Feature
Add an efficient interface for generation probabilities on fixed prompt and completion pairs. For example:
```python
# ... load LLM or engine
prompt_completion_pairs = [
…
-
**Describe the Feature**
Hi @shahules786 @jjmachan, I'm back in Ragas business ^^
I've recently stumble upon a `Failed to parse output. Returning None.` while trying to evaluate the faithfulness o…
-
# URL
- https://arxiv.org/abs/2409.12941
# Affiliations
- Satyapriya Krishna, N/A
- Kalpesh Krishna, N/A
- Anhad Mohananey, N/A
- Steven Schwarcz, N/A
- Adam Stambler, N/A
- Shyam Upadhyay…
-
i deifned this function to not use the LLM calls but it still give me this error:
Exception has occurred: AttributeError
'list' object has no attribute 'get_sample_type'
File "C:\Users\Amin\One…
-
I'm trying to follow https://github.com/karpathy/llm.c/discussions/481 but I'm getting this error:
```
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalua…
-
## Suggest for LLM Evaluation Tutorials with `Evalverse`
- **Tutorials** (Notebook examples): https://github.com/UpstageAI/evalverse/tree/main/examples
- [01_basic_usage.ipynb](https://github.com…
-
When I try to evaluate the quantized AWQ models using the video evalaution script, I'm getting FileNotFoundError.
```
FileNotFoundError: No such file or directory: "/hfhub/hub/models--Efficient-La…