search

Fus3n / infinite-sides

Infinite Craft but in Pyside6 and Python with local LLM (llama3 & others) using Ollama
MIT License
29 stars 2 forks source link

Error with koboldcpp #4

Open pbz134 opened 4 months ago

pbz134 commented 4 months ago

Describe the bug When trying to connect the program to koboldcpp's API (localhost:5001/v1) it throws an error.

To Reproduce Install infinite-sides as normal, then launch a .gguf model with koboldcpp and try to connect infinite-sides to the API address of koboldcpp.

Expected behavior infinite-sides should connect to koboldcpp and display the currently running model in the selection menu.

Screenshots grafik

Desktop (please complete the following information):

Fus3n commented 4 months ago

Currently Infinite sides only support ollama or any other software that supports ollama's API, so you might need to install ollama and models through it, i could add support for others but that might break some parts of the app. If you want custom models i think ollama has modelfiles you can check that out,