search

Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
20.53k stars 1.03k forks source link

Bug: ILL_ILLOPN when trying to run bartowski/DeepSeek-V2-Chat-0628-GGUF #538

Closed ELigoP closed 2 months ago

ELigoP commented 3 months ago

Contact Details

No response

What happened?

I am on Ubuntu 24.04 Threadripper 3970X. I also have 2x RTX 3090 GPUs. GGUF files are run fine with llama.cpp.

I run command

llamafile --n-gpu-layers 999 -m ~/.cache/lm-studio/models/bartowski/DeepSeek-V2-Chat-0628-GGUF/DeepSeek-V2-Chat-0628-Q4_K_M-00001-of-00004.gguf

and got error (--ftrace error log attached as a file):

$ llamafile --n-gpu-layers 999 -m ~/.cache/lm-studio/models/bartowski/DeepSeek-V2-Chat-0628-GGUF/DeepSeek-V2-Chat-0628-Q4_K_M-00001-of-00004.gguf 

error: Uncaught SIGILL (ILL_ILLOPN) at 0x534694 on qualcomm-builder pid 854399 tid 854399
  /usr/local/bin/llamafile
  No such file or directory
  Linux Cosmopolitan 3.6.2 MODE=x86_64; #40-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul  5 10:34:03 UTC 2024 qualcomm-builder 6.8.0-40-generic

RAX 0000000000000000 RBX 0000000000b2a3e4 RDI 0000000000b2a3e4
RCX 000000000000002d RDX 0000000000000001 RSI 0000000000534630
RBP 00007fff2e8c3e30 RSP 00007fff2e8c3e18 RIP 0000000000534694
 R8 73726579616c2d75  R9 73726579616c2d75 R10 00007a71f4ec2138
R11 00007a71f4ca7720 R12 0000000000000005 R13 00007fff2e8c86b8
R14 00007fff2e8c86e8 R15 00007fff2e8c8830
TLS 0000000000ae9dc0

XMM0  00ff0000ff000000ff00000000000000 XMM8  00000000000000000000000000000000
XMM1  00000000000000000000000000000000 XMM9  00000000000000000000000000000000
XMM2  ffffffff000002580000025800001f90 XMM10 00000000000000000000000000000000
XMM3  000000000000000000007a71f4ca8730 XMM11 00000000000000000000000000000000
XMM4  000000000000000000007a71f4ca7754 XMM12 00000000000000000000000000000000
XMM5  0000000000000000000000004b000000 XMM13 00000000000000000000000000000000
XMM6  000000000000001d0000000000000031 XMM14 00000000000000000000000000000000
XMM7  00000020000000000000000000000000 XMM15 00000000000000000000000000000000

cosmoaddr2line /usr/local/bin/llamafile 534694 5346c0 532a96 673636 4f4214 4f7e1c 401d40 433272 4015f4

0x000000000053468f: ?? ??:0
0x00000000005346c0: ?? ??:0
0x0000000000532a96: ?? ??:0
0x0000000000673636: ?? ??:0
0x00000000004f4214: ?? ??:0
0x00000000004f7e1c: ?? ??:0
0x0000000000401d40: ?? ??:0
0x0000000000433272: ?? ??:0
0x00000000004015f4: ?? ??:0

000000400000-000000adb1f8 r-x-- 7020kb
000000adc000-00000319a000 rw--- 39mb
0006fe000000-0006fe001000 rw-pa 4096b
7a71f4aae000-7a71f4b91000 rw-pa 908kb
7a71f4b91000-7a71f4c3225b r--s- 645kb
7a71f4c8d000-7a71f4c8e000 ---pa 4096b
7a71f4c8e000-7a71f4ca13e8 rw-pa 77kb
7a71f4ca2000-7a71f4ee3000 rw-pa 2308kb
7fff2e0cc000-7fff2e8cc000 rw--- 8192kb
# 60'252'160 bytes in 9 mappings
/usr/local/bin/llamafile --n-gpu-layers 999 -m /home/ai/.cache/lm-studio/models/bartowski/DeepSeek-V2-Chat-0628-GGUF/DeepSeek-V2-Chat-0628-Q4_K_M-00001-of-00004.gguf 
Illegal instruction (core dumped)

ftrace.log

Version

llamafile v0.8.12

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

jart commented 2 months ago

Try upgrading the llamafile-0.8.13