search

ngxson / wllama

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
https://huggingface.co/spaces/ngxson/wllama
MIT License
444 stars 23 forks source link

warning: munmap failed: Invalid argument #7

Closed flatsiedatsie closed 6 months ago

flatsiedatsie commented 7 months ago

I'm seeing this warning when loading a model:

............................................................................................warning: munmap failed: Invalid argument

Is it anything to worry about?

ngxson commented 7 months ago

That seems like a problem with mmap support in emscripten. I'm not sure where the problem comes from, probably I should see if there's a related issue in emscripten's repo: https://github.com/emscripten-core/emscripten/issues?q=munmap

In the meantime, could you please give more details on the model and configs that you're using?

flatsiedatsie commented 7 months ago

I'm using many models, but in this case it's https://huggingface.co/bartowski/h2o-danube2-1.8b-chat-GGUF/resolve/main/h2o-danube2-1.8b-chat-Q5_0.gguf with default settings.

flatsiedatsie commented 6 months ago

I believe this happens with every model I load.

felladrin commented 6 months ago

I also get this warning for any model I use.

It's also reproducible in the demo:

image
flatsiedatsie commented 6 months ago

Perhaps this is of interest, I came accross it by chance:

https://github.com/xenova/transformers.js/pull/545/files/0f58121096f1cf73e0b04b01d7c2a4e426a7822a..a4d88acdfadec970095734ef85f2879c521ef060

ngxson commented 6 months ago

I'm closing this issue because this error message is not very important (can be ignored)

Technical explanation: llama.cpp uses mmap() to read files, which may use less memory than traditional read() syscall. This benefits us because wllama also use mmap() to prevent copying files (see #39). However, we don't implement munmap() because we don't need to unload file. This is not needed because to unload a model, we can simply call wllama.exit()