-
It would be nice to be able to run ggufs on cpu like you can with llama gguf I don't know of what the speed would look like but could be better for people with low vram gpus
Also i haven't looked a…
-
If possible, could support for this model be added to ollama?
https://huggingface.co/allenai/OLMo-7B
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
Would it possible to implement 1.58 bit quantization on candle ? It was proposed in the following paper,
https://arxiv.org/pdf/2402.17764.pdf
The main inspiration behind using 1.58 bit implement…
-
Backtrace:
```
Call parameter type does not match function signature!
%StackGuardSlot = alloca ptr, align 8, addrspace(5)
ptr call void @llvm.stackprotector(ptr %8, ptr addrspace(5) %StackGu…
-
https://github.com/ggerganov/llama.cpp/pull/6454#issuecomment-2119832472
-
py-cord[voice] is already installed
D:\Anaconda3\envs\Comfyui\lib\site-packages\vector_quantize_pytorch\vector_quantize_pytorch.py:461: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated…
-
This will take some time, so this is just rough sketch for later:
- windows build/binary could expect `llama.dll`
- could be straightforward, because zig can cross-compile so I can do this local…
-
**LocalAI version:**
commit 3829aba869f8925dde7a1c9f280a4718dda3a18c/ docker 6102e12c4df1
**Environment, CPU architecture, OS, and Version:**
MacBook Air M2, Ventura 13.4
**Describe the …
-
Hello guys, i try to run mpt-7b model , and i am getting this code, i appreciate any help, here is the detail
Node.js v19.5.0
node_modules\llama-node\dist\llm\llama-cpp.cjs:82
this.inst…