-
Since the new GPTQ-for-LLaMa commits it is necessary to re-quantize the models to be compatible. Can someone upload them (because the ones from decapoda-research are too old and do not work)
-
```
~~> nix run .#textgen-nvidia -- --api --auto-launch
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
2023-12-12 23:44:37 ERROR:Could not fi…
-
### Feature request
just buttons to control the models from oobabooga like:
continue
replace last
remove last
edit response
copy last reply
then character customization or custom prompts that…
-
Hello, I am using oobabooga webui, and one of the scripts in that webui requires monkeypatch to be installed.
I attempted to install monkeypatch using "pip install monkeypatch", and got the following…
-
Need to switch to exllama, everything I'm reading about is how exllama is better. At least for production we will need to switch. Speed is everything at the inference volume we expect. Note to try VLL…
-
I ran the following script on the home page
python finetune.py ./data.txt \
--ds_type=txt \
--lora_out_dir=./test/ \
--llama_q4_config_dir=./llama-7b-4bit/ \
--llama_q4_model=./ll…
-
Think about what Automatic1111 did to Stable Diffusion, from a rather brute one-shot image generator significantly worse than the commercial counterparts it is now a distribution with thousands of fea…
-
The current API is about to be deprecated and will be replaced with an OpenAI compatible API on November 13th. This update will likely break oobabot so It needs to be updated.
-
### Describe the bug
Exllama v2 crashes when starting to load in the third gpu. No matter if the order is 3090,3090,A4000 or A4000,3090,3090, when I try to load the Mistral Large 2407 exl2 3.0bpw it …
-
Either locally or using a cloud-based service. I want to demo it using the least amount of effort possible : )