LM_Studio 0.2.31 on RHEL 8.10 no longer detects my GPU (RTX A4000)

[ElCoyote27]

Hi,

First of all, RHEL8 is enterprise Linux and is supported by Red Hat + major ISVs until mid-2029. (https://access.redhat.com/support/policy/updates/errata). This is not to say you -should- support LMStudio on RHEL8, too (LMStudio works fine on RHEL9; unfortunatelyI can't upgrade my fleet at the moment), but from an end-user point of view, it feels really close, and maybe it'll just take a few minor changes. As a small ISV, I understand it might be a difficult thing to do but perhaps it's only a few minor changes.

Now, onto the issue: On my workstation, LM_Studio-0.2.23-Ubuntu-20.04.AppImage starts up fine and includes this output:

10:12:50.096 › Backend-hardware compatibility survey complete:
{
  "llama_cuda" : {
    "libPaths": {
      "libLmStudioPath": "/tmp/.mount_LM_StunRnVqy/resources/app/.webpack/main/build/Release/CUDA/liblmstudio_bindings_cuda.node",
      "llmEngineLibPath": "/tmp/.mount_LM_StunRnVqy/resources/app/.webpack/main/build/Release/CUDA/llm_engine_cuda.node"
    },
    "isCompatible": true,
    "hardwareSurveyResult": {
      "gpuSurveyResult": {
        "result": {
          "code": "Success",
          "message": ""
        },
        "gpuInfo": [
          {
            "name": "NVIDIA RTX A4000",
            "deviceId": 0,
            "totalMemoryCapacityBytes": 16786456576,
            "integrationType": "Discrete",
            "detectionPlatform": "Cuda",
            "otherInfo": {}
          }
        ]
      },
      "cpuSurveyResult": {
        "result": {
          "code": "Success",
          "message": ""
        },
        "cpuInfo": {
          "architecture": "x86",
          "supportedInstructionSets": [
            "AVX2"
          ]
        }
      }
    }
  },

And then the main window comes up

However, if I start LM_Studio-0.2.31.AppImage, I get this output, the main window comes up and a notification window comes up which says: "compatibility issue Detected'

In the terminal I see this:

setConfiguration called but the number of loaded models is not 1.
10:18:57.229 › Current downloads folder: /export/home/raistlin/.cache/lm-studio/models
10:18:57.229 › One or more GPU names are undefined or GPU survey result is incomplete
[readJsonFile] Error reading file /export/home/raistlin/.cache/lm-studio/config-presets/config.map.json: SyntaxError: Unexpected end of JSON input
Error reading file /export/home/raistlin/.cache/lm-studio/config-presets/config.map.json: SyntaxError: Unexpected end of JSON input
[LMSAuthenticator][Client=LM Studio][LMSContext(ep=handle,t=channel)] Handling deep link
Error occurred in handler for 'hardware:get-current-gpu-backend-name': TypeError: Cannot read properties of null (reading 'get')
    at t.BackendManager.getCurrentBackendConfig (/tmp/.mount_LM_StuSZkGYr/resources/app/.webpack/main/index.js:8:676278)
    at /tmp/.mount_LM_StuSZkGYr/resources/app/.webpack/main/index.js:8:854711
    at node:electron/js2c/browser_init:2:98085
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:2:81603)
    at EventEmitter.emit (node:events:513:28)
Error occurred in handler for 'hardware:get-acceleration-available': h [Error]: Error getting backend options to check if GPU acceleration is available: Error: Cannot get best backend options when there are no backends available
    at /tmp/.mount_LM_StuSZkGYr/resources/app/.webpack/main/index.js:8:855053
    at node:electron/js2c/browser_init:2:98085
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:2:81603)
    at EventEmitter.emit (node:events:513:28) {

If you wanted to support RHEL8, I believe it would perhaps work better on 18.04 (it's super old, I know) or perhaps 16.04. Some other small ISVs (KeepassXC, Standard Notes) do that for their AppImages. nvidia-smi output:

$ nvidia-smi 
Sat Aug  3 10:23:43 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.58.02              Driver Version: 555.58.02      CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A4000               Off |   00000000:D9:00.0  On |                  Off |
| 41%   36C    P8             15W /  140W |     619MiB /  16376MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
[abbreviated]