search

SillyTavern / SillyTavern-Extras

Extensions API for SillyTavern.
GNU Affero General Public License v3.0
549 stars 124 forks source link

[Question] Any way to use a second GPU #18

Closed Manimap closed 1 year ago

Manimap commented 1 year ago

Hello,

I have my main gpu being totally used by loading a model, but I have a second gpu available locally : is it possible to ask the model to use the second gpu for all requests? If it's possible, is it doable on windows?

Cohee1207 commented 1 year ago

On line 134 of server.py, change the string of device from cuda:0 to cuda Please keep in mind that some modules will not work with multi-GPU setup and require a device index. If you encounter it - hard code the device passed to the model initialization to cuda:0

Manimap commented 1 year ago

Alright, I will try that, thank you.