Tracking: Webassembly support

[williballenthin]

Does zydis-rs already support one of the wasm targets? If not, what is the outlook for this?

I'll admit, I haven't researched this too thoroughly, so maybe the answer is obvious already. In any case, I'd like to express that I'm interested in using zydis-rs in a wasm project (probably some sort of binary analysis that runs in the browser). Therefore, if this has already been considered (and done?), then sample code or pointers to getting this working are appreciated.

[th0rex]

I don't think zydis builds for any webassembly target yet, at least I couldn't get it to build when I tried it. If it builds I don't think theres anything stopping the rust bindings from working with webassembly targets.

[williballenthin]

thank you!

if/when i give this a shot, i'll share the results.

[th0rex]

I've made two PRs to the zycore repo, fixing no libc builds and adding very hacky support for wasm32-unknown-wasi target (but it builds). If they're accepted I can push a commit in this repo which makes this crate work with wasm32-unknown-wasi and then someone can experiment with it, because I honestly do not know how to run .wasm files.

I don't even know if wasm32-unknown-wasi is the target you would want, or if you want to use wasm32-unknown-unknown, but both build for me so choose which ever is appropiate.

Edit: I've pushed a temporary webassembly branch, which checkouts all the submodules to my forks with the webassembly branch, you should be able to checkout that branch in this repo, then do git submodule update --init --recursive --checkout --force, get a libclang_rt.builtins-wasm32.a file (see here) and finally build with cargo run --example simple --features wasm --target wasm32-unknown-wasi for example.

I have absolutely no idea if there are better ways to do it and would appreciate help.

[th0rex]

I've added a python script and a test.html file that can load any of the examples with wasm32-unknown-unknown, however currently for any formatting related task webassembly complains about a function signature mismatch for the hooks in the Formatter. The pattern example seems to run without errors, but I don't know where the output of println! statements is supposed to go, I don't see any output in the console.

Edit: Seems like wasi is what handles the printing and so on, uploading pattern.wasm here (when compiled for wasm32-unknown-wasi) prints the expected output. I will in the next week try to figure out what the issue with the formatter hooks is and hopefully fix them. If/When they're fixed I think compiling to webassembly is working pretty well then, and we might be able to add official support.

Note that afaik compiling with emscripten is and has been working for a long time already, its used on the official zydis website after all, but I wasn't able to get the rust bindings to work with it, however I'm really inexperienced in anything web related.

[williballenthin]

Thanks for this effort!

I spent some time this afternoon trying to get zydis working with wasm. I had a number of false starts, but it seems that by using the latest nightly rust builds with clang 8.0.0, this can work!

I started with the sample rust+wasm project provided by mozilla, and added zydis-rs as a dependency. I think this is the best workflow that we should aim for (e.g. wasm-pack build and/or cargo build). The sample project is here: https://github.com/williballenthin/zydis-wasm . I'll add more documentation as I make a bit more progress.

still a couple things to chase down, but good progress so far!

[williballenthin]

I think that you should prefer wasm32-unknown-unknown over the -wasi target. WASI is a standard to enable wasm modules to run in a runtime outside the browser, sort of like the JVM. As zydis simply parses data and manipulates structures, I don't expect that it would reference file/network/process APIs - so no need for WASI interfaces. Also, -wasi isn't for the browser, which I believe is an interesting target for zydis.

I suspect if zydis works for -unknown then it will work for -wasi.

[williballenthin]

Regarding the emscripten build... that's my backup plan; however, if everything can be integrated into the rust ecosystem, then obviously things will be even nicer. For example, LTO.

I also have a lot to learn in order to help here, but I'll do my best. Thanks for your assistance so far!

[williballenthin]

but I don't know where the output of println! statements is supposed to go, I don't see any output in the console.

consider using the web-sys crate to access console.log: https://rustwasm.github.io/docs/book/game-of-life/debugging.html#add-logging-to-our-game-of-life

Seems like wasi is what handles the printing and so on

Well, maybe WASI is relevant to the browser, sorry for the confusion.

[williballenthin]

This morning I was able to build a Rust library depending on zydis-rs to WebAssembly. Here's what I've learned.

The most relevant thread is here: https://github.com/rustwasm/team/issues/291 In summary, linking Rust object files to C object files to produce a WebAssembly module is tricky. Today, rustc-generated object files probably uses a different ABI than a system's compiler (e.g. GCC or Clang). There might be some specific rust environments and setups that could work, like emscripten and wasi targets, but to me it almost seems like this is by chance, as they don't solve the underlying issues yet.

Back in May 2021, the posts Zig Makes Rust Cross-compilation Just Work and Zig Makes Go Cross Compilation Just Work demonstrated how to use Zig as an easy CC/CXX replacement to cross compile Rust projects with C dependencies. This works because Zig distributes a full LLVM build with most options enabled, including all the cross-compilation targets.

I was able to get this to work with zydis-rs fairly quickly. The zy* projects need a few small tweaks, which are contributed here:

• https://github.com/zyantific/zycore-c/pull/40
• https://github.com/zyantific/zydis/pull/277
• https://github.com/zyantific/zydis-rs/pull/28

They boil down to teaching zycore to detect the wasm target and zydis to build in NO_LIBC mode.

With these changes, I've updated https://github.com/williballenthin/zydis-wasm/ with the demonstration project.

[williballenthin]

As of today, I don't believe there's a simple Cargo configuration change to enable WebAssembly builds of zydis-rs. The Zig trick is most reasonable and can easily be done in CI by projects that rely on zydis-rs. Unfortunately it requires that little bit of extra configuration outside of Cargo. I'll update here if I notice that the Rust+WebAssembly ecosystem changes.

An alternative strategy I considered was to use c2rust to port zydis-c to pure Rust; however, there are many obvious downsides to this and I'm glad I didn't attempt it.

[msuiche]

Hi @williballenthin - thanks for sharing your research on this