-
Just updated my GPU from a 2080 to a 3090 and man does it makes things go brrrr lol.
Anyways, I notice a new strange behavor when I did. Instead of model + GPU taking close to what the model took i…
-
Good afternoon, first of all, thank you for your work. It is very interesting. I am a java developer in production so I don't have much expertise in python. When I run the example from your youtube, I…
-
I've been using this chatdocs project with a ggml model which has worked really well if a bit slow. I have read a lot online about GPTQ models delivering significantly better speeds, but when I triale…
-
### Describe the bug
When using the latest version after gguf merge (commit 0c9e818bb874df6429bf1c5fe96be6400516abd9), generated text appears to be missing some words.
Example:
> Common sense q…
-
GGML version from TheBloke is coming soon: https://huggingface.co/TheBloke/Llama-2-7B-GGML
Thanks!
-
Hi there. The llamacpp is now supporting q2 quantization. Is there any chance this comes to ctransformers?
-
How do I specify model() parameters, from here: https://github.com/marella/ctransformers#config
Placing them inside model() does not raise an error, but they are ignored for my mpt model ie. I can …
-
Operating System: Ubuntu 20.04.6 LTS
Kernel: Linux 5.4.0-146-generic
Architecture: x86-64
configs/local_config.yaml,shows
...
binding_name: llama_cpp_official
...
model_…
-
There is CLBlast GPU support for GPT-2 based models on koboldcpp for example, where I can do prompt processing on the GPU VRAM for less prompt batching errors with my 16GB of CPU RAM. Does anyone know…
-
```
Traceback (most recent call last):
File "~/git/lollms-webui/app.py", line 1668, in
user_avatar_path = lollms_paths.personal_user_infos_path / "default_user.svg"
AttributeError: 'LollmsP…