search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.21k stars 358 forks source link

Broken pipe error #1160

Open Asherathe opened 3 weeks ago

Asherathe commented 3 weeks ago

When using RisuAI 136.0.1 as the frontend with a Yi 34B based model, text generation will cut off after around 400-500 tokens. Terminal shows the error message: "Token streaming was interrupted or aborted! [Errno 32] Broken pipe". Streaming is enabled in Risu, max response tokens is set to 4096, and API selected is OpenAI-compatible.

Kobold.cpp is version 1.76 (although this also happened with the previous version), started with the command line "./koboldcpp-linux-x64-nocuda --usevulkan --gpulayers 24 --threads 7 --contextsize 8192".

On up-to-date openSuse Tumbleweed, i7-12700KF cpu with 32GB DDR4 RAM and AMD RX 6800 card.

LostRuins commented 3 weeks ago

I'm guessing the connection is being closed by the frontend

Asherathe commented 3 weeks ago

I'm guessing the connection is being closed by the frontend

Okay, I'll make a bug report there. Thanks.