-
i saw this error :
```
value is not a valid list (type=type_error.list))
Evaluating: 33%|█████████████████████████████▎ | 1/3 [01:37 0
v…
-
@haileyschoelkopf @lintangsutawika @baberabb
The following is a list of TODOs to implement LLM-as-a-Judge in Eval-Harness:
**TLDR**
* Splits existing `evaluate` function into `classification_e…
-
# Problem
prompt engineering is slow, and in a multi-agent system is very hard. You need to experiment with a message arbitrarily deep in the stack quickly. The current replay feature is helpful, but …
-
xtuner train cfg-gamma7b/gemma_7b_it_qlora_alpaca_e3_copy.py --deepspeed deepspeed_zero2
10/28 18:19:05 - mmengine - WARNING - WARNING: command error: ''Adafactor is already registered in optimizer a…
-
hi when i run demo.ipynb
with open('/root/llm-reasoners/examples/CoT/blocksworld/prompts/pool_prompt_v1.json') as f:
prompt = json.load(f) #/root/llm-reasoners/examples
evaluator …
-
Phoenix has two concepts of evaluators, those used in conjunction with `llm_classify` that are used to annotate spans, and those used as part of our experiments api to annotate dataset examples. Some …
-
### Describe the bug
So I want to evaluate a particular trace in which my answer and citations are both in the output.
So I gave the following prompt to it
 and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to use a local LLM in the evaluate function., whe…
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
Error 'openai.APIConnectionError: Connection error.' occurs w…
-
Current candidate
Mistral
https://mistral.ai/
RAG - LlamaIndex
https://github.com/mistralai/cookbook/blob/main/llamaindex_agentic_rag.ipynb
Can be tested out locally and seems to be a good ch…