search

bytecodealliance / wasmtime

A fast and secure runtime for WebAssembly
https://wasmtime.dev/
Apache License 2.0
15.45k stars 1.31k forks source link

add experimental support for illumos #9535

Closed sunshowers closed 3 weeks ago

sunshowers commented 3 weeks ago

With this change, the basic wasm tests all pass on illumos. Note the addition of NORESERVE to mmap calls.

However:

While wasmtime appears to be functional on illumos, it is still quite slow, particularly in the wast tests. For example, the following test:

cargo +beta test --test wast -- Cranelift/pooling/tests/spec_testsuite/load.wast

takes 0.07 seconds on Linux, but over 5 seconds on illumos. Some profiling suggests that this is due to lock contention inside the kernel while freeing memory, so I don't think this is a wasmtime issue. I'd like to pull some illumos experts in to do some debugging here as time permits, but I don't think this PR should necessarily be held up on that.

Thanks to iximeow for all the help with this!

(One note is that due to a rustc segfault on illumos, building wasmtime requires Rust 1.83 or higher. I did my building and testing with +beta.)

sunshowers commented 3 weeks ago

re slowness, for future reference, with 997Hz sampling:

user stacks: https://gist.github.com/sunshowers/b69b7bd2e671d9c23355d5e952636c5e

kernel stacks: https://gist.github.com/sunshowers/fa822f161e54d57a8103f6736656fbe8

sunshowers commented 3 weeks ago

Change default settings in Wasmtime for illumos (e.g. different defaults for the pooling allocator or something like that)

Interesting -- how would I go about doing this? I was floundering around the wasmtime code haha :)

sunshowers commented 3 weeks ago

One high-level question for you: how easy is it to compile for illumos from Linux? For example we'd ideally couple this with a check in our CI that illumos continues to build (e.g. at least via cargo check) but our CI only runs Linux/macOS/Windows right now so we'd have to cross-compile. If it's easy to cross-compile I think it'd be quite reasonable to have a check added to CI that this passes.

cargo check is definitely feasible. cargo build would require the linker etc, and is likely easier with cross.

edit: ahh spoke too soon. cargo check on illumos currently fails with "error occurred: Failed to find tool. Is gar installed?" (full output).

gar is what GNU ar is called on illumos. One could mangle paths and such but it's easier to just use cross build --target x86_64-unknown-illumos, which works great.

sunshowers commented 3 weeks ago

Ugh, I guess I need to clean the target dir for illumos. Silly glibc.

sunshowers commented 3 weeks ago

All right, CI updated and I think this is good to go.