-
**Describe the bug**
Following the instructions in https://github.com/microsoft/Olive/tree/main/examples/whisper, I carried out the following steps for model optimization:
1)
```
python3 prepare…
-
Would it be possible to add MMMU validation to EvalAI?
It'd be great to be able to compare the numbers calculated on the validation set with the ones produced by EvalAI.
-
## 🚀 Feature
allow f32q5_k and f16q5_k quantizations
## Motivation
from my tests, f16 (for output and embed tensors) and q5_k_m for the others is the best quantization.
## Alternatives
as…
-
**Describe the bug**
I use llama-2 7b, and when I start stage 2 in EE-Tuning, the bug occurs.
**To Reproduce**
here is `llama2_7B_1_exit_mlp_pt.sh` I modified:
``` bash
#!/bin/bash
PROJECT…
-
When using HFModel, the entire prompt is included in the output Prediction. It works as expected using Ollama
Sorry, I could not test it with the same model. My laptop is not able to run mistral-7B…
-
**Qwen2**
warning: not compiled with GPU offload support, --n-gpu-layers option will be ignored
warning: see main README.md for information on enabling GPU BLAS support
Log start
main: build = 2…
-
**Describe the bug**
If I prompt my chatflow agent with groq as llm and prompt it "what divsion equals to 1337?" the chat window shows the "thinking" animation, stop and does not provide an answer. O…
-
In the aviary cluster yaml file ,
image used is `anyscale/aviary:test`
should it be changed to `anyscale/ray-llm:latest`
```
# An unique identifier for the head node and workers of this cluster…
-
I am working on this project, and I want to use bedrock as I my chat service of choice, I tested this library with openai chatgpt-4, and it works perfectly.
Here is the code I am testing to connect…
-
Objective:
I have many test cases - query, response, context trios in a pandas dataframe. I create an llm test case per trio.. however the outcome of bulk testing locally is a json file.
Outcome:
…