[Feature request] Specify which GPU to use if CUDA is selected as a command line argument.

SillyTavern / SillyTavern-Extras

Extensions API for SillyTavern.

GNU Affero General Public License v3.0

563 stars 133 forks source link

[Feature request] Specify which GPU to use if CUDA is selected as a command line argument. #65

Closed Amajiro closed 1 year ago

Amajiro commented 1 year ago

I have two graphics cards and do my sillytavern-extras processing on the second one. Could we have a command line argument (maybe as a condition to --CUDA) to specify which GPU to use? Otherwise I have to edit device_string = "cuda:0" every time there's an update.

Cohee1207 commented 1 year ago

Added --cuda-device command line option. Defaults to cuda:0 as previously stated. Also, keep in mind that not every transformers module supports running in multi-GPU mode.