Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
19.39k stars 982 forks source link

Running llamafile on 2 GPU's instead of 1 #381

Open TheAmpPlayer opened 5 months ago

TheAmpPlayer commented 5 months ago

I am trying to run llamafile on windows and llamafiles uses both GPU's and limit's the VRAM to the weaker one. Is there a way to manually select on what GPU to run? I get incompatible error codes when it's trying to use both GPU's