search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

Allow to choose different HordeAi url for embedded worker #698

Closed Tomislaw closed 4 months ago

Tomislaw commented 6 months ago

Currently url is hardcoded. It would be great if we can change it via cmd args.

https://github.com/LostRuins/koboldcpp/blob/concedo/koboldcpp.py#L2095

LostRuins commented 6 months ago

Is there a new horde cluster I'm not aware of?

Tomislaw commented 6 months ago

No, I just experimenting with creating api wrapper for current horde, which introduces some extra logging and monitoring and I wanted to change url to localhost instead.

And If for some reason current horde url changes in the future then it might cause some issues.

(horde.koboldai.net already redirect requests to stablehorde.net, so my assumption isn't far-fetched )

LostRuins commented 6 months ago

I see, cool. it probably won't be a new launcher parameter, but it should be quite easy to change it within the python script as you highlighted.