-
Hi there,
I was testing this project and it worked perfectly with the default compose and Llama2 model, but when i use the compose-gpu model, i get no more a chat llm but a response that is a continu…
-
```
Python 3.10.11 (main, Apr 20 2023, 19:02:41) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.17.2 -- An enhanced Interactive Python. Type '?' for help.
I…
-
Here is my HF format of exllamav2 model
```python
import torch, os
from contextlib import contextmanager
from pathlib import Path
from typing import Optional, List, Union, Dict
from transforme…
-
## 🐛 Bug
## To Reproduce
Steps to reproduce the behavior:
1. I compiled this model: https://huggingface.co/Weyaxi/OpenHermes-2.5-neural-chat-7b-v3-1-7B
2. copy it onto my device with the…
-
**Bug description**
It looks like I'm consistently encountering an issue when using the `Researcher` role. At some point during the research, an SSL `ssl.SSLWantReadError` exception is encountered, a…
-
This thread is dedicated to discussing the setup of the webui on Intel Arc GPUs.
You are welcome to ask questions as well as share your experiences, tips, and insights to make the process easier fo…
-
I've been trying a few of the new mistral merges (Toten5/Marcoroni-neural-chat-7B-v1, OpenPipe/mistral-ft-optimized-1218) and am having the same problem where the original model works fine in transfor…
-
Hello,
I was trying to use kompute, I manage to compile llamacpp kompute with the follow steps
```
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp/
#install kompute repo…
-
I need my model name to match chatGPT's in order to integrate with an existing product.
To accomplish this by setting the 'served-model-name' option show below::
python -m vllm.entrypoints.opena…
-
**Motivation**
Instead of giving too many options in the list, we should best 1-2 best of the best models out of 5 best models in hardware range right now.
**Discussion**
I've made evaluation que…