search

WebAssembly / wasi-nn

Neural Network proposal for WASI
429 stars 34 forks source link

add: ggml graph encoding format #66

Closed danbev closed 3 months ago

danbev commented 4 months ago

This commit adds the ggml graph encoding format to the graph_encoding enum.

The motivation for this is to allow the wasi-nn interface to support models that are encoded in the ggml format which is the model format used by llama.cpp.

abrown commented 4 months ago

Thanks for the PR! I had been wondering when this new encoding would come up. Can you attend the ML working group on the 18th of this month (details)? It might be helpful for others interested in wasi-nn to understand the motivation here.

danbev commented 4 months ago

@abrown I'd be happy to attend if I can make the time slot :+1:

devigned commented 3 months ago

@danbev, what wasi-nn backend would be used to load and run the ggml encoded models? I might misunderstand, but I don't think any of the backends currently implemented will be able to load these models. Are you suggesting a new backend implementation?

danbev commented 3 months ago

what wasi-nn backend would be used to load and run the ggml encoded models.

WasmEdge has support for a llama.cpp backend. I have got a very basic llama.cpp backend working for Wasmtime and would very much like to see support for such a backend in Wasmtime in the future. Having this encoding would hopefully help existing implementations and creating new ones easier.

Are you suggesting a new backend implementation?

Yes, I'd be interested in seeing support for llama.cpp backend in Wasmtime (and other wasm runtimes).