[AVX] Nitro exited with code: null

zdavatz commented 3 months ago

Download the Appimage for Linux with Kernel 6.8.0
Start the App
./jan-linux-x86_64-0.4.10.AppImage
Download Mistral Instruct 7B Q4
Do a query

Then on my Gentoo Linux Box I get:


Update for version 0.4.10 is not available (latest version: 0.4.10, downgrade is disallowed).
2024-04-03T08:18:50.832Z [NITRO]::CPU informations - 4
2024-04-03T08:18:50.833Z [NITRO]::Debug: Request to kill Nitro
2024-04-03T08:18:50.883Z [NITRO]::Debug: Nitro process is terminated
2024-04-03T08:18:50.884Z [NITRO]::Debug: Spawning Nitro subprocess...
2024-04-03T08:18:50.885Z [NITRO]::Debug: Spawn nitro at path: /home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/bin/linux-cpu/nitro, and args: 1,127.0.0.1,3928
2024-04-03T08:18:50.893Z [NITRO]::Debug: Nitro exited with code: null
2024-04-03T08:18:50.893Z [NITRO]::Error: child process exited with code null
(node:13858) UnhandledPromiseRejectionWarning: Error: timeout
at Timeout.timeoutFunc (/home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/node/index.cjs.js:3074:25)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7)
(Use `jan --trace-warnings ...` to show where the warning was created)
(node:13858) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
2024-04-03T08:20:30.068Z [NITRO]::CPU informations - 4
2024-04-03T08:20:30.069Z [NITRO]::Debug: Request to kill Nitro
2024-04-03T08:20:30.072Z [NITRO]::Debug: Nitro process is terminated
2024-04-03T08:20:30.073Z [NITRO]::Debug: Spawning Nitro subprocess...
2024-04-03T08:20:30.073Z [NITRO]::Debug: Spawn nitro at path: /home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/bin/linux-cpu/nitro, and args: 1,127.0.0.1,3928
2024-04-03T08:20:30.083Z [NITRO]::Debug: Nitro exited with code: null
2024-04-03T08:20:30.083Z [NITRO]::Error: child process exited with code null
Error: Loader Message: setup_loader_term_phys_devs:  Failed to detect any valid GPUs in the current config
Warning: vkEnumeratePhysicalDevices
at GatherPhysicalDevices (../../third_party/dawn/src/dawn/native/vulkan/VulkanInfo.cpp:144)
at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:388)
at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:301)
at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:556)

(node:13858) UnhandledPromiseRejectionWarning: Error: timeout at Timeout.timeoutFunc (/home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/node/index.cjs.js:3074:25) at listOnTimeout (node:internal/timers:569:17) at process.processTimers (node:internal/timers:512:7) (node:13858) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)


7. Do I need a GPU to run this?
8. This is disabled
![latest-screenshot](https://github.com/janhq/jan/assets/4953/e74d3fe5-81f4-4cc5-8752-42e9060d8389)

Van-QA commented 3 months ago

hi @zdavatz, can you try this? https://jan.ai/docs/troubleshooting#troubleshooting-nvidia-gpu

zdavatz commented 3 months ago

Thanks!

Do I really have to turn on GPU support?

My GPU card is pretty old: NVIDIA Corporation GP107

Isn't this a Nitro problem?

Installing Nitro also fails:

~> curl -sfL https://raw.githubusercontent.com/janhq/nitro/main/install.sh | sudo /bin/bash -
Failed to fetch latest version.

Van-QA commented 3 months ago

hi @zdavatz, you can still run using the CPU run mode, try turning off this GPU Acceleration setting:

zdavatz commented 3 months ago

That is the case, see point 8 above in the issue.

Van-QA commented 3 months ago

I see, the issue might have been resolved using nightly build, would you mind trying it? @zdavatz https://github.com/janhq/jan?tab=readme-ov-file#download

zdavatz commented 3 months ago

Error still looks similar:

Error occurred in handler for 'readFileSync': Error: ENOENT: no such file or directory, open '/home/zeno/jan/settings/@janhq/inference-openai-extension/settings.json'
    at Object.openSync (node:fs:603:3)
    at Object.func [as openSync] (node:electron/js2c/asar_bundle:2:2238)
    at Object.readFileSync (node:fs:471:35)
    at t.readFileSync (node:electron/js2c/asar_bundle:2:9777)
    at /tmp/.mount_jan-liuXKXwB/resources/app.asar/node_modules/@janhq/core/dist/node/index.cjs.js:2170:35
    at async WebContents.<anonymous> (node:electron/js2c/browser_init:2:86716) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/home/zeno/jan/settings/@janhq/inference-openai-extension/settings.json'
}
Error occurred in handler for 'readFileSync': Error: ENOENT: no such file or directory, open '/home/zeno/jan/settings/@janhq/inference-triton-trt-llm-extension/settings.json'
    at Object.openSync (node:fs:603:3)
    at Object.func [as openSync] (node:electron/js2c/asar_bundle:2:2238)
    at Object.readFileSync (node:fs:471:35)
    at t.readFileSync (node:electron/js2c/asar_bundle:2:9777)
    at /tmp/.mount_jan-liuXKXwB/resources/app.asar/node_modules/@janhq/core/dist/node/index.cjs.js:2170:35
    at async WebContents.<anonymous> (node:electron/js2c/browser_init:2:86716) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/home/zeno/jan/settings/@janhq/inference-triton-trt-llm-extension/settings.json'
}
2024-04-04T09:14:59.722Z [NITRO]::CPU informations - 4
2024-04-04T09:14:59.723Z [NITRO]::Debug: Request to kill Nitro
2024-04-04T09:14:59.771Z [NITRO]::Debug: Nitro process is terminated
2024-04-04T09:14:59.772Z [NITRO]::Debug: Spawning Nitro subprocess...
2024-04-04T09:14:59.773Z [NITRO]::Debug: Spawn nitro at path: /home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/bin/linux-cpu/nitro, and args: 1,127.0.0.1,3928
2024-04-04T09:14:59.781Z [NITRO]::Debug: Nitro exited with code: null
2024-04-04T09:14:59.782Z [NITRO]::Error: child process exited with code null
(node:24120) UnhandledPromiseRejectionWarning: Error: timeout
    at Timeout.timeoutFunc (/home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/node/index.cjs.js:3074:25)
    at listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)
(node:24120) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)

Van-QA commented 3 months ago

Sorry @zdavatz, can you give us more information about your hardware / specs / OS? here is our requirement for running Jan

zdavatz commented 3 months ago

Gentoo Linux with Kernel 6.8.0
lspci output: lspci.txt
AFAICT my box has all the requirements.

louis-jan commented 2 months ago

@zdavatz. Could I ask you to open /home/zeno/jan/extensions/@janhq/inference-nitro-extension/dist/bin/linux-cpu/nitro manually and duplicate the outputs here, please?

zdavatz commented 2 months ago

sure, I get:

~/jan/extensions/@janhq/inference-nitro-extension/dist/bin/linux-cpu> ./nitro 
Ungültiger Maschinenbefehl

zdavatz commented 2 months ago

my cat /proc/cpuinfo looks like this:

processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Core(TM) i7 CPU         960  @ 3.20GHz
stepping        : 5
microcode       : 0x11
cpu MHz         : 3200.000
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
initial apicid  : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm pti tpr_shadow flexpriority ept vpid dtherm ida vnmi
vmx flags       : vnmi preemption_timer invvpid ept_x_only flexpriority tsc_offset vtpr mtf vapic ept vpid
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips        : 6400.40
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

louis-jan commented 2 months ago

cc @tikikun @CameronNg @vansangpfiev

louis-jan commented 2 months ago

Update: Invalid CPU Instructions

louis-jan commented 2 months ago

cc @namchuai - AVX support

louis-jan commented 2 months ago

Ref #2598 #2729

janhq / jan

[AVX] Nitro exited with code: null #2595