search

lmstudio-ai / .github

34 stars 3 forks source link

Unable to load any models on Linux #40

Closed ericlay closed 1 month ago

ericlay commented 2 months ago

I have been unable to get any models to load when testing on Linux. I do not have the same issues on Windows 10/11 (works great there). Even on the same device.

I have tried on 3 separate devices with varying OS:

Desktop 1: Debian 12.5/AMD Ryzen5-5600x/32G/RX6600 Desktop 2: Windows 10/Intel Core i7-6700/16G Laptop: Dual boot - Windows 11 + Arch/Intel Core i7-1255u/16G/Intel Arc 370

Installed by: Windows - using the installer package on lmstudio.ai / Arch - using the AUR package / Debian - downloading the appimage and running from terminal ala ./

I don't have any issues running under Windows on either device but run into the same issue on both Arch and Debian. Unused ram is always reported as ~24-25KB, while various tools are showing ~18G free ram.

Latest error on Debian desktop (sometimes the code is 11 other times 0):


{
  "cause": "(Exit code: 11)?. Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "27.30 GB",
      "ram_unused": "28.25 KB"
    },
    "gpu": {
      "type": "AmdOpenCL",
      "vram_recommended_capacity": 0,
      "vram_unused": 0
    },
    "os": {
      "platform": "linux",
      "version": "6.6.15-amd64",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.19",
      "downloadsDir": "/home/eric/.cache/lm-studio/models"
    },
    "model": {}
  },
  "title": "Error loading model."
}```
SKajto commented 2 months ago

The same issue:

{
  "cause": "(Exit code: 0)?. Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "125.67 GB",
      "ram_unused": "127.67 KB"
    },
    "gpu": {
      "type": "AmdOpenCL",
      "vram_recommended_capacity": 0,
      "vram_unused": 0
    },
    "os": {
      "platform": "linux",
      "version": "6.1.0-1036-oem",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.20",
      "downloadsDir": "/home/skajto/.cache/lm-studio/models"
    },
    "model": {}
  },
  "title": "Error loading model."
}

System Info:

peterxie1 commented 2 months ago

I saw the same issue, cannot load any models with LM studio 0.2.18 on Ubuntu20.04. i saw error: "(exit code: 42)?. Please check settings and try loading the model again. " Anyone get it working on Linux?

thinkORo commented 2 months ago

I had an older version (0.2.16) up and running. Since 0.2.18 (today 0.20.0) LM studio cannot load any model.

JAvilaSP commented 2 months ago

Same problem here, Ubuntu 22.04.4


{
  "cause": "(Exit code: 0)?. Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "13.48 GB",
      "ram_unused": "15.39 KB"
    },
    "gpu": {
      "type": "AmdOpenCL",
      "vram_recommended_capacity": 0,
      "vram_unused": 0
    },
    "os": {
      "platform": "linux",
      "version": "6.5.0-28-generic",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.20",
      "downloadsDir": "/home/javila/.cache/lm-studio/models"
    },
    "model": {}
  },
  "title": "Error loading model."
}
aijdsofttech commented 2 months ago

I have a the same error in Ubuntu 20.04 LTS

nitishpatel commented 2 months ago

"memory": { "ram_capacity": "14.94 GB", "ram_unused": "24.25 KB" }, "gpu": { "type": "AmdOpenCL", "vram_recommended_capacity": 0, "vram_unused": 0 }, "os": { "platform": "linux", "version": "6.5.0-27-generic", "supports_avx2": true }, "app": { "version": "0.2.20", "downloadsDir": "/home/ntsh/.cache/lm-studio/models" }, "model": {} }

hrules6872 commented 2 months ago

Same issue but on Mac :( 

{
  "cause": "(Exit code: 42)?. Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "32.00 GB",
      "ram_unused": "10.00 GB"
    },
    "gpu": {
      "type": "AppleMetal",
      "vram_recommended_capacity": "21.33 GB",
      "vram_unused": "8.96 GB"
    },
    "os": {
      "platform": "darwin",
      "version": "14.4.1"
    },
    "app": {
      "version": "0.2.20",
      "downloadsDir": "/.cache/lm-studio/models"
    },
    "model": {}
  },
  "title": "Error loading model."
}
sobvan commented 2 months ago

I deleted LM Studio, downloaded it again from the site, and it worked. I think the auto update could have messed something up.

d-d commented 2 months ago

I deleted LM Studio, downloaded it again from the site, and it worked. I think the auto update could have messed something up.

Something is going with the installation process.

I had the same issue (error 42) on Mac. A fresh install from brew. Seeing this comment, I just reinstalled

brew reinstall --cask lm-studio

and it worked!

yagil commented 2 months ago

Error 42 on Mac was caused by an issue in our build system, sorry about that. If you redownload from the website it should work: https://lmstudio.ai

peterxie1 commented 2 months ago

Error 42 on Mac was caused by an issue in our build system, sorry about that. If you redownload from the website it should work: https://lmstudio.ai

Does this fix also help get it working on Ubuntu Linux? Thanks.

manu-git-code commented 2 months ago

Still getting this on Linux (PopOs) on app version 0.2.21. I've tried a couple of models. (Sometimes it returns an exit code 139 instead of zero, but the rest of the json is the same). 


{
  "cause": "(Exit code: 0). Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "30.50 GB",
      "ram_unused": "50.50 KB"
    },
    "gpu": {
      "type": "AmdOpenCL",
      "vram_recommended_capacity": 0,
      "vram_unused": 0
    },
    "os": {
      "platform": "linux",
      "version": "6.8.0-76060800daily20240311-generic",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.21",
      "downloadsDir": "/media/ai_models"
    },
    "model": {}
  },
  "title": "Error loading model."
}```
roooychen commented 2 months ago

I encountered (exit code: 139) when I set the wrong hardware setting. If you don't have a GPU card, just uncheck "Hardware Settings > GPU Acceleration > GPU Offload".

manu-git-code commented 2 months ago

I encountered (exit code: 139) when I set the wrong hardware setting. If you don't have a GPU card, just uncheck "Hardware Settings > GPU Acceleration > GPU Offload".

That worked! I've an AMD 7700XT, I guess it is not supported at the moment.

Thanks :)

emreds commented 2 months ago

I encountered (exit code: 139) when I set the wrong hardware setting. If you don't have a GPU card, just uncheck "Hardware Settings > GPU Acceleration > GPU Offload".

Also worked for me with an AMD GPU. Thanks a lot!

haraldschilly commented 2 months ago

Dumb question: where are these "Hardware Settings"? I'm on Linux, maybe they aren't shown?

emreds commented 2 months ago

Dumb question: where are these "Hardware Settings"? I'm on Linux, maybe they aren't shown?

On the AI Chat section, there is a Settings Sidebar on the right. Click the Advanced Settings in Settings Sidebar and you will see. It took me a while to find it.

haraldschilly commented 2 months ago

Dumb question: where are these "Hardware Settings"? I'm on Linux, maybe they aren't shown?

On the AI Chat section, there is a Settings Sidebar on the right. Click the Advanced Settings in Settings Sidebar and you will see. It took me a while to find it.

Ahh, thank you! It's buried in "Advanced Configuration" :face_with_spiral_eyes:

aijdsofttech commented 1 month ago

Facing the Same Issue with 20.04 LTS Ubuntu LM_Studio-0.2.21.AppImage I have GPU and Also it's Detected By Studio then what's the problem.

  "memory": {
    "ram_capacity": "125.57 GB",
    "ram_unused": "127.57 KB"
  },
  "gpu": {
    "type": "NvidiaCuda",
    "vram_recommended_capacity": "47.33 GB",
    "vram_unused": "46.97 GB"
  },
  "os": {
    "platform": "linux",
    "version": "5.15.0-105-generic",
    "supports_avx2": true
  },
  "app": {
    "version": "0.2.21",
    "downloadsDir": "/home/wp/.cache/lm-studio/models"
  },
  "model": {}

image

ericlay commented 1 month ago

I encountered (exit code: 139) when I set the wrong hardware setting. If you don't have a GPU card, just uncheck "Hardware Settings > GPU Acceleration > GPU Offload".

Oh wow, embarrassed I posted this without attempting that. This indeed does work on both Arch and Debian installs for me.

Closing now as this may just be pebcak.

mateitudose commented 1 month ago

This solution does not appear to be optimal, as the inference process on the CPU is significantly slower. I suggest reopening the issue to get a fix to also allow loading on the GPU

peterxie1 commented 1 month ago

I saw the same issue, cannot load any models with LM studio 0.2.18 on Ubuntu20.04. i saw error: "(exit code: 42)?. Please check settings and try loading the model again. " Anyone get it working on Linux?

on 0.2.18, errors when loading ANY model:
json
{
  "cause": "(Exit code: 42)?. Please check settings and try loading the model again. ",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "251.39 GB",
      "ram_unused": "253.39 KB"
    },
    "gpu": {
      "type": "Unknown",
      "vram_recommended_capacity": 0,
      "vram_unused": 0
    },
    "os": {
      "platform": "linux",
      "version": "5.15.47+prerelease4757",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.18",
      "downloadsDir": "/home/gta/.cache/lm-studio/models"
    },
    "model": {}
  },
  "title": "Error loading model."
}
========================================================
on 0.2.20 and 0.2.22 the LM Studio GUI does not even load, it crashed with the below:
A JavaScript error occurred in the main process
Uncaught Exception:
Error: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /tmp/.mount_LM_Stu6pC27n/resources/app/.webpack/main/build/Release/NoGPU/liblmstudio_bindings.node)
    at process.func [as dlopen] (node:electron/js2c/asar_bundle:2:1822)
    at Module._extensions..node (node:internal/modules/cjs/loader:1326:18)
    at Object.func [as .node] (node:electron/js2c/asar_bundle:2:1822)
    at Module.load (node:internal/modules/cjs/loader:1096:32)
    at Module._load (node:internal/modules/cjs/loader:937:12)
    at f._load (node:electron/js2c/asar_bundle:2:13330)
    at Module.require (node:internal/modules/cjs/loader:1120:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at c (/tmp/.mount_LM_Stu6pC27n/resources/app/.webpack/main/index.js:2:2765)
    at 21587 (/tmp/.mount_LM_Stu6pC27n/resources/app/.webpack/main/index.js:8:650139)
yagil commented 1 month ago

Ubuntu 22.04 is the minimum required for the Linux version. We hope to support older Ubuntu as well, but that’s not in place right now.

PixelOfDeath commented 1 month ago

I hat the issue (0.2.22 on Xubuntu 23.10), that GPU mode is selected at first but causes the (Error code: 0) when trying to load a model.

I unselect "GPU Acceleration" and then it loads the models just fine.

But if I open a new chat and try to load (or reload) a model I get that error again. I have to then select AND unselect "GPU Acceleration" again to make it work in the context of that new chat.

peterxie1 commented 1 month ago

Ubuntu 22.04 is the minimum required for the Linux version. We hope to support older Ubuntu as well, but that’s not in place right now.

Thanks. I upgraded from Ubuntu 20.04 to Ubuntu 22.04 and LM Studio 0.2.22 appears to be working! it can load TheBloke's Mistral Instruct v0.2 7B Q4_K_S gguf model with no issue. Thanks again.

javapapo commented 1 month ago

I agree with the above.

For example ollama- does manage to identify GPU capabilities on my thinkpad X13 (no matter if its not a very powerful GPU).

OS: Pop!_OS 22.04 LTS x86_64
GPU: AMD ATI 33:00.0 Rembrandt

this is the output from a fresh update of ollama locally on the same machine

>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
>>> Downloading AMD GPU dependencies...
######################################################################## 100.0%##O#-#               ######################################################################## 100.0%
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
>>> AMD GPU dependencies installed.

It seems that LMStudio can not detect these capabilities on certain flavours of Ubuntu or Debian based machines - not sure.

Archana-M-123 commented 1 month ago

I hat the issue (0.2.22 on Xubuntu 23.10), that GPU mode is selected at first but causes the (Error code: 0) when trying to load a model.

I unselect "GPU Acceleration" and then it loads the models just fine.

But if I open a new chat and try to load (or reload) a model I get that error again. I have to then select AND unselect "GPU Acceleration" again to make it work in the context of that new chat.

Thank u its working fyn

redet-G commented 1 week ago

I bootstrapped lms and disabled gup by running the following command. It is now working fine.

lms load --gpu=0