-
With the initial text generation metric PR, if an LLM provides an invalid response for one of our LLM guided metrics (wrong formatted, wrong data type, etc.), then Valor will raise an error and the re…
-
When using llm-foundry for model evaluation, multi-gpu mode does not work.
The source code is here: https://github.com/mlfoundations/open_lm/blob/main/eval/eval_openlm_ckpt.py
-
Lastly i was looking at the `YAML` files for the **UnifyAI** tools, and I had a few ideas that might help:
**Here are my thoughts 🤔**:
- `evaluate_llm_tool.yaml:`
It seems like the `prompt…
-
## Summary
This template is intended to capture a few base requirements that are needed to be met prior to filing a PR that contains a new blog post submission.
Please fill out this form in its…
-
Evaluation failed: 'CustomOllama' object has no attribute 'set_run_config', what is the solution,
Ragas Version: 0.1.7
**Code Examples**
# Define a simple dataset using Pandas DataFrame
data…
-
**Describe the bug**
I want to use local llms to evaluate my rag app, I have tried Ollama and HuggingFace models but neither of them is working.
Ragas version: 0.1.11
Python version: 3.11.3
**…
-
## Description
To enhance Inbox Zero's capability in handling PDF documents, particularly receipts and potentially more complex documents like pitch decks, we need to research and implement effective…
-
### Feature Description
The most popular LLMs such as OpenAI support candidate generations which means to generate n responses for the same prompt. This feature can be used in RAG, evaluations and mo…
-
I've been working on evaluating how well LLMs can handle bioimaging tasks relative to the complexity of the task.
First, we can see that different tasks have different probabilities of being easily…
-
I need code with llamaindex using bearer token and base url not with langchain.
from langchain_community.vectorstores import FAISS
from langchain_community.vectorstores import Chroma
from langcha…