search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.34k stars 310 forks source link

Just an appreciation post - the new colab is AWESOME! #484

Open teddybear082 opened 8 months ago

teddybear082 commented 8 months ago

Feel free to delete this just want to say THANK YOU! I love the new google colab for koboldcpp you created. It is super easy to get up and running and now I am powering the Skyrim AI mod in VR without losing performance, with responses from the best LLM role playing 13B models, with the same response time as ChatGPT and openrouter. This is amazing.

LostRuins commented 8 months ago

Yep. The Tiefighter model is excellent yeah? Feel free to recommend it to others!

teddybear082 commented 8 months ago

Thank you!! I also tried with mythomax 13b gguf k_m right now and it is great too. I ran both the Herika skyrim mod and Mantella mod with it for about 25 minutes right now, perfection. Best experience I've had since I can use my full computer performance to drive the game and not lose frames from the AI generation, so I get to bump up from 7B models to 13B AND have a better experience. This is so great.

heraclitus1212 commented 1 month ago

Hey, teddybear082, idk if you'll see this 6 months later but can you maybe tell me what I'm doing wrong in setting up Herika with Colab? I have been copying the Kobold API link into the dwemer distro and adding the correct port. I don't get an error message when testing my connection to the Herika server, but the LLM does not respond in the test or in game and no output appears in the Colab either. Sorry for the random ask just at my wit's end here.

teddybear082 commented 1 month ago

Hey, teddybear082, idk if you'll see this 6 months later but can you maybe tell me what I'm doing wrong in setting up Herika with Colab? I have been copying the Kobold API link into the dwemer distro and adding the correct port. I don't get an error message when testing my connection to the Herika server, but the LLM does not respond in the test or in game and no output appears in the Colab either. Sorry for the random ask just at my wit's end here.

Hi have you joined the Herika discord? If not I would check there for troubleshooting. I’m a little rusty at this point as I’ve been away from Skyrim for a few months. I do think I remember maybe something like :8080 or :429 had to be added to the link the server gives you but I forget the details.