search

codeproject / CodeProject.AI-Server

CodeProject.AI Server is a self contained service that software developers can include in, and distribute with, their applications in order to augment their apps with the power of AI.
https://codeproject.github.io/codeproject.ai
Other
730 stars 159 forks source link

Multi-GPU #158

Open kltye opened 1 week ago

kltye commented 1 week ago

Is there a way to force which GPU CPAI runs on? I've used CUDA_VISIBLE_DEVICES in Windows but it doesn't seem to have any effect (models appear to run on the GPU I wish to exclude). Additionally, does CPAI load-balance between GPUs?

m411b commented 1 week ago

I'd like to know the same. Is there a way to get CPAI to run on a chosen GPU rather than a GPU it picks?