search

Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
20.53k stars 1.03k forks source link

Bug: Llamafiler SIGSEGV crash #521

Closed BorisELG closed 3 months ago

BorisELG commented 3 months ago

Contact Details

elgarehb@gmail.com

What happened?

First, thank you for your work and all your improvments. I try to run llamafiler with mxbai-embed-large-v1-f16.gguf model provided here mixedbread-ai/mxbai-embed-large-v1.

Here, the command executed :

./llamafile-0.8.12/bin/llamafiler -m /opt/model/mxbai-embed-large-v1-f16.gguf -l 0.0.0.0:9000

But, after some requests, the server crash and it look like stuck :

2024-08-02T11:56:20.131319 llamafile/server/signals.cpp:57 198.18.134.129 crashed SIGSEGV at 0 bt 75a2b3 4207ae 4217d9 41ff1b 4201f8 41ed96 76dfa7 779044 7cc92f
2024-08-02T11:56:20.131994 llamafile/server/worker.cpp:51 198.18.133.69 all threads active! dropping oldest client

The same model work well with the old llamafile server method : /usr/bin/llamafile --server --nobrowser --embedding --host 0.0.0.0 --port 9000 -m /opt/model/mxbai-embed-large-v1-f16.gguf

Version

llamafile v0.8.12

What operating system are you seeing the problem on?

Linux

Relevant log output

2024-08-02T11:56:20.131319 llamafile/server/signals.cpp:57 198.18.134.129 crashed SIGSEGV at 0 bt 75a2b3 4207ae 4217d9 41ff1b 4201f8 41ed96 76dfa7 779044 7cc92f
2024-08-02T11:56:20.131994 llamafile/server/worker.cpp:51 198.18.133.69 all threads active! dropping oldest client
ofer1992 commented 3 months ago

hey, tried to reproduce it on my M1 but couldn't get a crash.

BorisELG commented 3 months ago

Resolved with the v0.8.13, the server doesn't crash when all threads are active and return a 503 HTTP status code.

jart commented 3 months ago

Hooray!