-
It would be fantastic if you added the ability to use this solely on the PC without the need for the flask. I saw the note and thought I'd oblige.
on a side note, I love the way your mind works.
J…
-
### What is the issue?
OS: Linux 6.5.0-44-generic #44~22.04.1-Ubuntu
GPU:
AMD Radeon RX 7900 XTX (24 GiB VRAM)
AMD Radeon RX 7900 XTX (24 GiB VRAM)
AMD Radeon RX 7900 XTX (24 GiB VR…
-
Would be good to have a comparison chart like one on this page https://github.com/MeetKai/functionary
jkfnc updated
2 months ago
-
### **Describe the bug**
for tensor shape (1,32,12,100) transpose doesn't work for -2, -3 dimensions. Namely, it throws an error that shape of the tensor must be divisible by tile dim. It seems that …
-
### Expected Behavior
A few days ago, I used the flux nf4 model to generate an image in just 1 minute.
### Actual Behavior
![Snipaste_2024-08-22_10-27-48](https://github.com/user-attachments/assets…
-
since I compiled for using cuda core
first I had to add `nGpuLayers` (seem logic as it's an option available in llama.cpp)
then I obtain this error:
```
TypeError: this.instance.inference is n…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
IMAGE SYNC
-
>[!NOTE]
>Until this is fixed the workaround is use the CPU or CUDA instead.
### Bug Report
Vulkan: Meta-Llama-3.1-8b-128k slow generation.
When using release 3.1.1 and Vulkan the Meta-Llama-3…
-
## 端测rag向量数据库
Gemma2 2B 才刚掀起了端侧模型热,端侧 RAG 就来了!🔥🤯
C 语言实现,基于 SQLite 插件的「最快⚡」端侧向量数据库 sqlite-vec 开源,短时间暴涨 1.8K Star⭐
- 处理50万个 960 维向量仅 41 毫秒
- 支持 JS/Rust,支持 Llama.cpp 离线Embedding 和在线Embedding
…