Are there any benchmark programs for Wasm GC circulating around?

[fitzgen]

Hey folks! I've finally got all of Wasm GC implemented for Wasmtime and all spec tests passing.[^1] I also have a sizable backlog of potential optimizations to do. But, of course, it seems pretty silly to just randomly chip away at that backlog; much better would be to guide my efforts based on the patterns toolchains are actually emitting and what Real World programs using Wasm GC actually look like.

So: are there any Wasm GC programs circulating around that I can use for some quick/initial benchmarking? Ideally with minimal imports and Web/JS assumptions. j2cl? kotlin? dart? scheme? ocaml?

Thanks!!

[^1]: At the time of writing, there is one final open PR that hasn't merged into main yet, just in case anyone immediately runs off to try running some Wasm GC program and it fails.

[kripken]

The main benchmarks I've looked at for WasmGC all depend on imported JS strings (and other JS APIs, but strings is the hardest to replace). Is there some kind of polyfill for JS strings for non-JS environments? If so then those might be ported perhaps.

[peteryongzhong]

The Chrome team implied that they ported the Fannkuch benchmark to WasmGC. However, I am not sure if that effort is open source.

https://developer.chrome.com/blog/wasmgc#porting_programming_languages_to_new_runtimes_with_wasmgc

[kripken]

@peteryongzhong That is open source, yes: basically just compile the Java Fannkuch benchmark with J2Wasm. There are other popular benchmarks in this space that are available too (e.g. we used Box2D, DeltaBlue, RayTrace, and Richards in a blogpost). However, J2Wasm does use JS Strings as I mentioned before, which would be a blocker here unless someone has figured out a polyfill for that for non-Web environments.

[fitzgen]

Thanks for the links! I'll look into hacking up the benchmarks to avoid strings...

[bashor]

@fitzgen, we have a bunch of micro and macro benchmarks at https://kotl.in/wasm-benchmarks. But another challenge could be that compilers using the GC proposal also use the EH proposal.

Fortunately, for experimental(!) purposes, we added an option to compile without using the EH proposal. In this mode, any throw will lead to a trap, so it's not for every application. We didn't check to run benchmarks with this mode, but you/we can try.

[fitzgen]

@bashor thanks, sounds very promising! FWIW, we are hoping to implement exceptions Soon as well.

[vouillon]

At the moment, the code generated by Wasm_of_ocaml relies quite a lot on JavaScript. But I was thinking about hacking the runtime support to get rid of this dependency, at least for simple benchmarks, to be able to test implementations of the Stack Switching proposal (https://github.com/ocaml-wasm/wasm_of_ocaml/pull/52). We use exceptions as well.

[tlively]

Dart has some benchmarks available as well: https://github.com/mkustermann/wasm_gc_benchmarks

[zapashcanon]

If you are interested, we have some benchmark programs generated by Wasocaml that do not require the EH proposal and do not rely a lot on JS. You only have to provide an equivalent of the following:

  "print_i32": print_i32,
  "print_f64": print_f64,
  "print_endline": print_endline,
  "putchar": putchar,
  "flush": flush,
  "atan2": Math.atan2,
  "sin": Math.sin,
  "asin": Math.asin,
  "fmod": (x, y) => x % y,
  "cos": Math.cos,

It is also interesting because we are comparing to OCaml native, OCaml Bytecode, and OCaml compiled to JS (through js_of__ocaml), but also wasm_of_ocaml. They all have different behaviors and are interesting to compare to:

[tlively]

cc @danleh