search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.13k stars 81 forks source link

crash on GGML_ASSERT: 'rwkv.cpp/ggml/src/ggml.c:5316: ggml_can_repeat_rows(b, a)' #138

Closed bigcat26 closed 9 months ago

bigcat26 commented 9 months ago

System: Linux x86_64, Ubuntu 22.04.3 LTS

python3 python/generate_completions.py RWKV-5-World-0.1B-v1-20230803-ctx4096-ggml.bin 
System info: AVX=1 AVX2=1 AVX512=0 FMA=1 NEON=0 ARM_FMA=0 F16C=1 FP16_VA=0 WASM_SIMD=0 BLAS=0 SSE3=1 VSX=0
Loading RWKV model
GGML_ASSERT: rwkv.cpp/ggml/src/ggml.c:5316: ggml_can_repeat_rows(b, a)
Aborted (core dumped)

gdb

Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350497344) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350497344) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737350497344) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737350497344, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7cca476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7cb07f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007fffd30baea4 in ggml_add_impl (ctx=0x7fffd31d6908 <g_state+104>, a=0x7fffc03ffad0, b=0x7fffb8753e90, inplace=false)
    at /work/rwkv.cpp/ggml/src/ggml.c:5316
#6  0x00007fffd30bafdb in ggml_add (ctx=0x7fffd31d6908 <g_state+104>, a=0x7fffc03ffad0, b=0x7fffb8753e90)
    at /work/rwkv.cpp/ggml/src/ggml.c:5340
#7  0x00007fffd30f83c0 in rwkv_att_wkv (ctx=0x7fffd31d6908 <g_state+104>, att_time_first=0x7fffc03ffad0, att_time_decay=0x7fffc03ff950, 
    k=0x7fffb8753e90, v=0x7fffb8753fe0, aa=@0x7fffffffcf20: 0x7fffb87486d0, bb=@0x7fffffffcf28: 0x7fffb8748820, 
    pp=@0x7fffffffcf30: 0x7fffb8748970) at /work/rwkv.cpp/rwkv_graph.inc:132
#8  0x00007fffd30f86fc in rwkv_att (ctx=0x7fffd31d6908 <g_state+104>, x=0x7fffb8752ae0, layer=..., state=...)
    at /work/rwkv.cpp/rwkv_graph.inc:172
#9  0x00007fffd30f90e3 in rwkv_build_serial_graph (model=..., graph=...) at /work/rwkv.cpp/rwkv_graph.inc:274
#10 0x00007fffd30f9552 in rwkv_measure_and_build_serial_context (model=..., graph=...) at /work/rwkv.cpp/rwkv_graph.inc:320
#11 0x00007fffd30fa6db in rwkv_init_from_file (
    file_path=0x7fffd37e7770 "/work/models/RWKV-5-World-0.1B-v1-20230803-ctx4096-ggml.bin", n_threads=4)
    at /work/rwkv.cpp/rwkv.cpp:65
#12 0x00007ffff7083052 in ffi_call_unix64 () from /.conda/envs/pytorch/lib/python3.11/lib-dynload/../../libffi.so.8
#13 0x00007ffff7081925 in ffi_call_int () from /.conda/envs/pytorch/lib/python3.11/lib-dynload/../../libffi.so.8
#14 0x00007ffff708206e in ffi_call () from /.conda/envs/pytorch/lib/python3.11/lib-dynload/../../libffi.so.8
#15 0x00007ffff70932e4 in _call_function_pointer (argtypecount=<optimized out>, argcount=2, resmem=0x7fffffffd1a0, restype=<optimized out>, 
    atypes=<optimized out>, avalues=<optimized out>, pProc=0x7fffd30fa471 <rwkv_init_from_file(char const*, uint32_t)>, flags=4353)
    at /usr/local/src/conda/python-3.11.4/Modules/_ctypes/callproc.c:923

llama.cpp works fine.

saharNooby commented 9 months ago

H!

RWKV-5-World-0.1B-v1-20230803-ctx4096-ggml.bin

RWKV 5 is not supported yet. I have plans to support it, but can't say when it will be done.