search

distantmagic / paddler

Stateful load balancer custom-tailored for llama.cpp 🏓🦙
MIT License
632 stars 27 forks source link

Req for windows Build #8

Closed jkfnc closed 4 months ago

jkfnc commented 4 months ago

Thanks for making this, is it possible to include windows builds as part of releases ?

mcharytoniuk commented 4 months ago

@jkfnc Will do. :) Follow this issue to stay posted. I will get to it this week.

mcharytoniuk commented 4 months ago

@jkfnc I have added Windows and MacOS binaries to the latest Paddler release. From now on, those OSs will be included by default in Paddler releases. :)

If you need any other OS to be included, either let me know or make a PR and add it to the build matrix: https://github.com/distantmagic/paddler/blob/main/.github/workflows/release.yml

https://github.com/distantmagic/paddler/releases/tag/v0.3.0

jkfnc commented 4 months ago

Thanks, working good. But the management-server crashed two times, will try to replicate next time, is there way to enable debug level logs.

mcharytoniuk commented 4 months ago

@jkfnc If you want, we can debug that together.

I recently covered multi-GPU setup issues in v4.0.0 release. There might be some issues under Windows as well.

If you can spend some time on it, you can share your screen on Discord, and we can debug that case, too.

Alternatively - I am rewriting some concurrency code for 5.0. I might let you know when that is ready instead.

mcharytoniuk commented 4 months ago

@jkfnc I just released v0.5.0. I checked it against all races and potential threading issues I could find. I don't know exactly what caused the specific issue you mentioned, but there is a chance that v0.5.0 won't crash under Windows anymore.

jkfnc commented 4 months ago

Thanks, will try it out and let you know..