How to compile Virgil into WASI

[syrusakbary]

I've been playing with the idea of getting compilers and new languages into the Wasm/WASI world, as I really think it enriches the ecosystem. Zig has recently supported compiling to Wasm/WASI and I was wondering how we could get the self-hosting Virgil compiler into the WASI world.

I tried playing a bit with the /bin/dev bash files but I was unable to get Virgil compiling into WASI (the compiler itself). Any help will be greatly appreciated!

[titzer]

You can try this:

% cd aeneas/src
% v3c-wasi -shadow-stack-size=1m -heap-size=500m -output=/tmp */*.v3 $VIRGIL_LOC/lib/util/*.v3 $VIRGIL_LOC/lib/asm/*/*.v3

The generated .wasm file does run, but file loading is broken atm, so it can't load any files yet. Not very useful for a compiler, I know :-)

[titzer]

Also a very worthwhile option is -rt.symtab, which will generate a names section.

[syrusakbary]

I was able to get it compiling, but not fully working (I guess because the file issue you commented). We can keep this issue open until aenea.wasm is able to fully work in a Wasm/WASI context :)

Not sure if this may help or not, but usually when having WASI applications you may want to provide them with a --dir=. or --mapdir=.:/ (for example) so the runtime can access files/directories on that dir.

[syrusakbary]

I'd love to play a bit more on this. I got aenea.wasm but nothing outputted once I ran it with a wasm runtime. Any advice on how to continue this effort forward? What's pending to have? (any guidance would be welcome!)

[titzer]

Hmm, that's weird. If I compile Aeneas with v3c-wasi and run that Aeneas.wasm on node, I get:

[~/virgil]% v3c-wasi $AENEAS_SOURCES
[~/virgil]% node --no-warnings --experimental-wasi-unstable-preview1 rt/wasi_snapshot_preview1/wasi.node.mjs Aeneas.wasm -help
Aeneas III-6.1505
Usage: v3c (-run|-test|-version|-target=<target>) [options] <v3 files>

LANGUAGE OPTIONS

    -legacy-cast[=(true|false)]
        Enable legacy cast semantics (int.! == int.view) for the Virgil language.

SHARED OPTIONS

    -opt=<string>
        Set optimization configuration options.
    -maxr=<int>
        Limit the maximum number of return values in normalization.
    -maxp=<int>
        Limit the maximum number of parameters in normalization.
    -maxd=<int>
        Set the maximum number of data slots allowed for auto-unboxing data types.
...

This also works in wizard, but doesn't work in wasm3. That seems to be because wasm3 passes the the module name as the first argument to a module (which causes Aeneas to not recognize further options).

[syrusakbary]

That seems to be because wasm3 passes the the module name as the first argument to a module (which causes Aeneas to not recognize further options).

This is a common practice in most server-side Wasm/WASI runtimes. I believe Wasmer, wasmtime, WasmEdge, and wasm3 all do that (this is, so the binary can recognize the program being run). Could it be possible to support this in Virgil?

This would imply that wasmer Aeneas.wasm or wasm3 Aeneas.wasm (or other runtime) would work without any modifications in the runtime itself

[titzer]

Done. I fixed the RiRuntime.v3 for WASI to skip the first argument. (Also fixed wizard to pass the .wasm filename as the first argument). This works now in node, wasm3, and wizard. Didn't try wasmer yet.

[titzer]

What remains is for me to fixup fileLoad() in rt/wasi_snapshot_preview1/System.v3, and I think the compiler will work. Mostly the issue is just getting the file size properly and loading the data into the array.

[syrusakbary]

Thanks for the fix. I recompiled and I verified this actually works with wasm3 (via wax wasm3 Aeneas.wasm).

However, I can't get it to work with Wasmer or Wasmtime. I'm really not sure what's going on at first sight. I'm inclined to think it might be a small bug on the WASI impl in Virgil, since the behavior seen in Wasmer or Wasmtime is the same: no output. I assumed the first argument thing will fix it, but doesn't seem to be the case.

I inspected the module and everything seems fine compared to other WASI modules that run properly in all runtimes.

$ wasmer inspect Aeneas.wasm
Type: wasm
Size: 1.9 MB
Imports:
  Functions:
    "wasi_snapshot_preview1"."args_sizes_get": [I32, I32] -> [I32]
    "wasi_snapshot_preview1"."args_get": [I32, I32] -> [I32]
    "wasi_snapshot_preview1"."path_open": [I32, I32, I32, I32, I32, I64, I64, I32, I32] -> [I32]
    "wasi_snapshot_preview1"."fd_close": [I32] -> [I32]
    "wasi_snapshot_preview1"."fd_seek": [I32, I64, I32, I32] -> [I32]
    "wasi_snapshot_preview1"."fd_write": [I32, I32, I32, I32] -> [I32]
    "wasi_snapshot_preview1"."fd_read": [I32, I32, I32, I32] -> [I32]
    "wasi_snapshot_preview1"."clock_time_get": [I32, I64, I32] -> [I32]
  Memories:
  Tables:
  Globals:
Exports:
  Functions:
    "_start": [] -> [I32]
  Memories:
    "memory": not shared (8020 pages..8020 pages)
  Tables:
  Globals:

What remains is for me to fixup fileLoad() in rt/wasi_snapshot_preview1/System.v3

Awesome, I'd really love to get it fully working!

[syrusakbary]

Oh, I noticed one small difference. Other WASI modules (generated with Rust), always call proc_exit at the end of main with the desired exit code.

Imports:
  Functions:
    "wasi_snapshot_preview1"."proc_exit": [I32] -> []

While the Wasm/WASI generated module (Aeneas.wasm), doesn't even import this function. It might be the case that the stdout is not even flushed?

[titzer]

Yeah, I noticed that wasmtime sometimes doesn't flush stdout. Maybe calling proc_exit will fix it; I plan to add that to the RiRuntime interface for the wasm targets, as this is the proper contact point between the compiler and the runtime system.

[titzer]

https://github.com/titzer/virgil/commit/8a0d527c26bc6cfd7e960f2250437f19f11bfc5f

[syrusakbary]

Small update: I just tested the latest master with the commit you pointed @titzer, but it still doesn't work with wasmer or wasmtime.

I'm now a bit confused on what's going on :/

[syrusakbary]

I tried to take on the challenge again!

Here's how to try it locally!

git clone https://github.com/titzer/virgil.git
cd virgil

export VIRGIL_LOC=$(pwd)
export PATH=$PATH:$(pwd)/bin/:$(pwd)/bin/dev

# make sure that java is available in your path
v3c-wasi -shadow-stack-size=1m -heap-size=500m -output=/tmp */*.v3 $VIRGIL_LOC/lib/util/*.v3 \
$VIRGIL_LOC/lib/asm/x86-64/X86_64Assembler.v3 \
$VIRGIL_LOC/lib/asm/arm/ArmAssembler.v3 \
$VIRGIL_LOC/lib/asm/x86/X86Assembler.v3 \
$VIRGIL_LOC/lib/asm/mips/MipsAssembler.v3

The file is now being generated properly: /tmp/Aeneas.wasm.

However, I'm not sure how to run it with Wasmer, I'm trying the following but it doesn't work:

git clone https://github.com/titzer/virgil.git
cd virgil

cd apps/HelloWorld
wasmer run /tmp/Aeneas.wasm --dir=. -- HelloWorld.v3

But it outputs:

File not found: HelloWorld.v3

Any hints on what I may be doing wrong @titzer ?

[titzer]

I think that the problem is in rt/wasi_snapshot_preview1/System.v3 that doesn't use WASI file operations properly. I had meant to debug that, but I've gotten sidetracked. It's one of the more annoying things about WASI, IMHO, that all file system operations have to be relative to an open directory and then that file descriptor number must be coordinated with the embedding.