Closed daniandtheweb closed 1 year ago
It should be possible to set the CLBLAST_PLATFORM
environment variable to achieve this, but I haven't tested it.
There are two environment variables you can set: GGML_CLBLAST_PLATFORM
and GGML_CLBLAST_DEVICE
(just numbers)
(Personally I think there should be a command line argument, and it's not really CLBlast that it's controlling, but OpenCL in general. CUDA doesn't have any way to control the GPU selection right now, @slaren?)
CUDA doesn't have any way to control the GPU selection right now, @slaren?
The environment variable CUDA_VISIBLE_DEVICES
should work.
Thanks @SlyEcho for the info. I also think that a command line argument would be the best but I guess that for now I can just close the issue.
Using CLBlast on some devices (like my Radeon 5700XT) the program defaults to device id 0 and platform id 0, that causes the following error:
When I manually edit the platform id inside the source to 1 however makes the program behave as expected using the GPU for the prompt evaluation:
My suggestion is to add a launch option to manually choose the desired platform and device id similarly to the --useclblast option of koboldcpp.