Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
20.62k stars 1.04k forks source link

Bug: mlock is failing and llama-server is outdated for very long time. #591

Closed BoQsc closed 1 month ago

BoQsc commented 1 month ago

Contact Details

No response

What happened?

mlock doesn't work at all and webui of llamafile --server clearly shows that llama.cpp llama-server is very outdated in this release of llamafile

Version

llamafile-0.8.14

What operating system are you seeing the problem on?

Windows

Relevant log output

warning: failed to mlock 43745280-byte buffer (after previously locking 0 bytes): Out of memory (win32 error 1453)
BoQsc commented 1 month ago

Very outdated. image

BoQsc commented 1 month ago

It seems like llamacpp of llamafile is so old, it does not support mlock on Windows that is virtuallock.

jart commented 1 month ago

We haven't updated the llama.cpp server since they removed vision support upstream. We're currently planning on replacing it with our own new server.