Support for RPC Backend

leejet / stable-diffusion.cpp

Stable Diffusion and Flux in pure C/C++

MIT License

3.28k stars 276 forks source link

Support for RPC Backend #392

Open ecyht2 opened 2 weeks ago

ecyht2 commented 2 weeks ago

Add support for RPC backend for GGML. This allows for distributed inferencing on multiple GPU over the network. Not so important now as the current models are still kinda small but would be interesting to have this feature.

Green-Sky commented 2 weeks ago

First we need partial/split offloading. (but yea, rpc would be one of the usable backends)