-
### Describe the bug
attempting to load a model results in the error ERROR:Failed to disable exllama. Does the config.json for this model contain the necessary quantization info?
### Is there an exi…
-
### Before submitting your bug report
- [X] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue](ht…
-
### Describe the bug
I am running a Gradio application locally, where there's a requests request to a remote server in the click event function of a button, and the result is returned to the componen…
-
Hi,
I want to try connecting Jarvis using oobabooga/text-generation-webui as a backend instead of openai for private and offline solution. Is that possible? Thanks!
-
Unsure if this is an exllamav2 issue or a llama-cpp issue. (In contrast, GGUF Q8_0 conversion of BF16 worked.)
When I loaded it via ooba/llama-cpp, inference broke when context length exceeded 4K, al…
-
### 🐛 Describe the bug
When trying to use pytorch-nightly there is no cuda Version in version.py generated file
```
File "G:\git-jv\ai\oobabooga_one-click-installersk-installers\installer_fil…
-
I cannot use taskweaver on api hosted by oobabooga/text-generation-webui. However, it work with llama cpp model loader on oobabooga/text-generation-webui which I cannot use it well as AMD rocm 5.7 has…
-
**Is your feature request related to a problem? Please describe.**
Ollama and cloud APIs support simple text generation (continuations) in addition to chat, but this is not exposed in WebUI.
**Des…
-
### The Feature
Currently users have to manually override "**base_url**" in their code when using a local API server. LiteLLM should support "**local/**" in the beginning of the model name which wo…
-
## 🚀 Feature
It would be nice to get a `/v1/models` REST endpoint with a behaviour similar to [OpenAI](https://platform.openai.com/docs/api-reference/models/list) or [oobabooga/Text-Generation-WebUI]…