-
### OS
Windows
### GPU Library
CUDA 12.x
### Python version
3.10
### Pytorch version
3.10.8
### Model
turboderp/Llama-3.1-8B-Instruct-exl2
### Describe the bug
I always receive `assistant` …
-
environment: wsl inside Windows.
```
$ free -m
total used free shared buff/cache available
Mem: 23919 580 22559 3 7…
-
Stumbled across the repo and was interested in trying out the assistant concept with some beefed up local model settings.
My env is quite different than what's used here and my goal was to document…
-
Running log of build failure for github.com/oobabooga/text-generation-webui=1.14.0
-
Add a UI element for SD request params.
-
Couldn't make a PR for some reason, so here is a slightly modified version of script.py with added support for Searx.
```
# ----------------------------------------------------------------------…
-
# Trending repositories for C#
1. [**dotnet / runtime**](https://github.com/dotnet/runtime)
__.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.__
…
-
### Describe the bug
It seems impossible to update the path (or add new paths) for Oobabooga to load models from.
As a result, a user would have multiple copies of the same model on their machine w…
-
Hello Team,
We want to run litserve on a single machine. The catch being we want to load models from S3.
The model file path and the model name needs to be read from a config file.
Want to understa…
-
### Describe the bug
The latest dev branch is not able to load any gguf models, with either llama.cpp or llamacpp_hf loader.
### Is there an existing issue for this?
- [X] I have searched the exist…