GPU Selection + Model Data Location

[nPHYN1T3]

Upon first launch I started digging through the options. I was looking to find a few key things. Two of those things being data location and GPU selection.

I eventually found if I was downloading a model I could choose a new location but then it never loads from that location unless you download it again. Manually moving the original download to the new location causes a conflict "freak out."

As for GPU select there doesn't appear to be such a thing. Worse is it seems to ignore all my GPU's and run solely on my CPU.

Running version LM_Studio-0.2.27.AppImage

[yagil]

Hi @nPHYN1T3 thanks. We'll investigate the location issue.

Did you mean select which GPU among the available GPUs? That's not yet supported.

But you should be able to select % offload to your GPU(s) vs CPU using the --gpu option in lms load

Example: full GPU offload

lms load --gpu=max

More info:

$ lms load --help
lms load
> Load a model

OPTIONS:
 ...
  --gpu <0-1|off|auto|max>  - How much to offload to the GPU. If "off", GPU offloading is disabled. If "max", all layers are offloaded to GPU. If "auto", LM Studio will decide how much to offload to the GPU. If a number between 0 and 1, that fraction of layers will be offloaded to the GPU. [optional]

[nPHYN1T3]

I initially wanted to reply "So then how does IT select a GPU?" but it looks like it's using them all in parallel...so gud!

OK I got the lms command going but seems like a lot of this is UI issues. The options are really crammed into that side bar and scrolling is finicky constantly scrolling sub sections and then the whole thing making it really hard to see/find/interact with things.

I saw there was a GPU slider in there (no GPU selection bit though) but when I tried it it tossed out a warning and said go low...which seems to just dump work on the CPU.

Thanks for the cli heads up. I had poked through the docs but the tiny "topic per page" makes it really hard to just search everything for any place the info you want might be.