search

second-state / WasmEdge-WASINN-examples

Apache License 2.0
217 stars 36 forks source link

ggml on macOS ARM64 support #31

Closed tudi2d closed 7 months ago

tudi2d commented 8 months ago

Really interested in this example & would love to play around with it, but I am unfortunalley not experienced with WasmEdge yet. I wanted to start this example locally on my machine (Apple Silicon) & ran into the following problem

Executed: curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- --plugins wasi_nn-ggml

Expected: Go through & install WasmEdge as well as the plugin

Actual: Installed WasmEdge, but threw an error for the specified plugin

Screenshot 2023-09-08 at 17 18 23

I did run the suggested source .../.zshenv command as well - without any luck

Alternative solution: I would also be happy to just run it in Docker, but I'm not really sure if that is possible with this plugin. Is there maybe any documentation on that? Would that be possible without being able to build the example?

juntao commented 8 months ago

I do not think we have ggml plugin for Mac in the release assets yet. You can build it yourself though. But Mac needs special compiler flags in order to take advantage of the Apple neural chips.

Please give us a couple of days to clean it up. Thanks!

tudi2d commented 7 months ago

Thanks for picking up this work & documenting the future work in this issue https://github.com/WasmEdge/WasmEdge/issues/2886! It should be fixed by https://github.com/WasmEdge/WasmEdge/pull/2871 when using the latest version. Will give it a try & re-open in case I run into an issue.