-
I am trying to run below code from @tomaarsen's HuggingFace blog on Sentence-Transformer-V3
### Code:
```python
from datasets import load_dataset
data = load_dataset('sentence-transformers…
-
Hi,
Are you planning making textgrad llm calls asynchronous?
I tried to start adding saynchronous methods to make at least evaluation calls and inference (everything that is forward) asynchrono…
-
I have some questions about the submission format. Do we need to include the conversation template of the LLM in the submission? For example, should the submission be formatted like
```
[INST] How t…
-
[X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
ValueError: Unknown format code 'f' for object of type 'str'
…
-
I'm using the promptfoo for a customized LLM. I would like to use the options on its webserver (Using the New Eval tab to run the evaluation).
Is there any solution to add my customised provider to …
-
To reproduce:
* Create a test set without a correct_answer column
* Run an evaluation
The evaluation will fail and the result will not be seen
Expected behavior:
The evaluation will fail, opening t…
-
### Project description
This project aims to collect a dataset of production COBOL and associated mainframe languages (JCL, REXX, PL/I) which Large Language Models (LLMs) can be fine-tuned on. It a…
-
Hi, this is a really good and useful codebase. I tried to reproduce the results reported in the paper but failed. I used the code in `README_ESE.md`:
```
WANDB_MODE=disabled CUDA_VISIBLE_DEVICES=0…
-
- [ ] [WisdomShell/kieval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models](https://github.com/WisdomShell/kieval)
# WisdomShell/kieval: A Knowledge-grounded Interacti…
-
# Proposed Feature
Add an efficient interface for generation probabilities on fixed prompt and completion pairs. For example:
```python
# ... load LLM or engine
prompt_completion_pairs = [
…