Bytebox wasm backend

[rdunnington]

These changes add support for the bytebox runtime to the wasm layer as an optional backend. The wasm backend still defaults to wasm3. To enable support, you must:

1. When building the SDK and tools, invoke orcadev build --wasm-backend bytebox.
2. When bundling the app, pass --wasm-backend bytebox

This change also includes some misc fixes:

• Update build scripts to preserve the runtime orca.dll's corresponding pdb, which was being overwritten by the orca.exe (tool) pdb.
• orca bundle: avoid printing error on Windows when attempting to remove temporary dir that didn't exist (would happen if you didn't supply an icon)
• orca bundle: instead of linking dynamically with runtime.obj, use a prebuilt orca_runtime.exe and directly embed the supplied icon file into it. This avoids requiring MSVC be in the path when running bundle commands.

[martinfouilleul]

I get these errors when trying to build with ./orcadev build --wasm-backend bytebox:

zig build-lib bytebox Debug native: error: the following command failed with 3 compilation errors:
/usr/local/bin/zig build-lib /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/src/cffi.zig --cache-dir /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/zig-cache --global-cache-dir /Users/martinfouilleul/.cache/zig --name bytebox -static --listen=- 
steps [3/16] zig build-exe bytebox Debug native... Semantic Analysis [2942] ...zig build-lib fibonacci Debug wasm32-freestanding: error: warning(link): unexpected LLD stderr:
wasm-ld: warning: creating shared libraries, with -shared, is not yet stable

zig build-lib add-one Debug wasm32-freestanding: error: warning(link): unexpected LLD stderr:
wasm-ld: warning: creating shared libraries, with -shared, is not yet stable

zig build-exe bytebox Debug native: error: the following command failed with 10 compilation errors:
/usr/local/bin/zig build-exe /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/run/main.zig --cache-dir /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/zig-cache --global-cache-dir /Users/martinfouilleul/.cache/zig --name bytebox --mod bytebox::/Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/src/core.zig --deps bytebox --listen=- 
zig build-lib mandelbrot Debug wasm32-freestanding: error: warning(link): unexpected LLD stderr:
wasm-ld: warning: creating shared libraries, with -shared, is not yet stable

zig build-exe testsuite Debug native: error: the following command failed with 3 compilation errors:
/usr/local/bin/zig build-exe /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/test/main.zig --cache-dir /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/zig-cache --global-cache-dir /Users/martinfouilleul/.cache/zig --name testsuite --mod bytebox::/Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/src/core.zig --deps bytebox --listen=- 
zig build-exe benchmark Debug native: error: the following command failed with 3 compilation errors:
/usr/local/bin/zig build-exe /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/bench/main.zig --cache-dir /Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/zig-cache --global-cache-dir /Users/martinfouilleul/.cache/zig --name benchmark --mod bytebox::/Users/martinfouilleul/Prog/Projects/Orca/src/ext/bytebox/src/core.zig --deps bytebox --listen=- 
Build Summary: 7/16 steps succeeded; 4 failed (disable with --summary none)
install transitive failure
├─ install bytebox transitive failure
│  └─ zig build-exe bytebox Debug native 10 errors
├─ install benchmark transitive failure
│  └─ zig build-exe benchmark Debug native 3 errors
├─ install bytebox transitive failure
│  └─ zig build-lib bytebox Debug native 3 errors
└─ install testsuite transitive failure
   └─ zig build-exe testsuite Debug native 3 errors
src/vm_stack.zig:4979:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/vm_stack.zig:5051:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/definition.zig:2687:37: error: no field or member function named 'readIntBig' in 'io.GenericReader(*io.fixed_buffer_stream.FixedBufferStream([]const u8),error{},(function 'read'))'
            const magic = try reader.readIntBig(u32);
                              ~~~~~~^~~~~~~~~~~
/usr/local/lib/zig/std/io.zig:117:12: note: struct declared here
    return struct {
           ^~~~~~
referenced by:
    decode: src/definition.zig:2635:13
    run: bench/main.zig:26:19
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
src/vm_stack.zig:4979:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/vm_stack.zig:5051:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
/usr/local/lib/zig/std/math.zig:91:21: error: Deprecated: use `nan(f32)` instead
pub const nan_f32 = @compileError("Deprecated: use `nan(f32)` instead");
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    parseF32: test/main.zig:210:32
    parseVal: test/main.zig:262:37
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
src/vm_stack.zig:4979:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/vm_stack.zig:5051:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/definition.zig:2687:37: error: no field or member function named 'readIntBig' in 'io.GenericReader(*io.fixed_buffer_stream.FixedBufferStream([]const u8),error{},(function 'read'))'
            const magic = try reader.readIntBig(u32);
                              ~~~~~~^~~~~~~~~~~
/usr/local/lib/zig/std/io.zig:117:12: note: struct declared here
    return struct {
           ^~~~~~
referenced by:
    decode: src/definition.zig:2635:13
    bb_module_definition_decode: src/cffi.zig:205:21
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
src/vm_stack.zig:4979:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/vm_stack.zig:5051:21: error: invalid builtin function: '@fabs'
        const abs = @fabs(vec);
                    ^~~~~~~~~~
src/definition.zig:2687:37: error: no field or member function named 'readIntBig' in 'io.GenericReader(*io.fixed_buffer_stream.FixedBufferStream([]const u8),error{},(function 'read'))'
            const magic = try reader.readIntBig(u32);
                              ~~~~~~^~~~~~~~~~~
/usr/local/lib/zig/std/io.zig:117:12: note: struct declared here
    return struct {
           ^~~~~~
referenced by:
    decode: src/definition.zig:2635:13
    main: run/main.zig:243:15
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
src/wasi.zig:1633:49: error: no field or member function named 'readIntLittle' in 'io.GenericReader(*io.fixed_buffer_stream.FixedBufferStream([]u8),error{},(function 'read'))'
                    const iov_base: u32 = reader.readIntLittle(u32) catch {
                                          ~~~~~~^~~~~~~~~~~~~~
/usr/local/lib/zig/std/io.zig:117:12: note: struct declared here
    return struct {
           ^~~~~~
src/wasi.zig:1633:49: error: no field or member function named 'readIntLittle' in 'io.GenericReader(*io.fixed_buffer_stream.FixedBufferStream([]u8),error{},(function 'read'))'
                    const iov_base: u32 = reader.readIntLittle(u32) catch {
                                          ~~~~~~^~~~~~~~~~~~~~
/usr/local/lib/zig/std/io.zig:117:12: note: struct declared here
    return struct {
           ^~~~~~
src/wasi.zig:1396:27: error: no field or member function named 'writeIntLittle' in 'io.writer.Writer(*io.fixed_buffer_stream.FixedBufferStream([]u8),error{NoSpaceLeft},(function 'write'))'
                    writer.writeIntLittle(u64, cookie) catch break;
                    ~~~~~~^~~~~~~~~~~~~~~
/usr/local/lib/zig/std/io/writer.zig:10:12: note: struct declared here
    return struct {
           ^~~~~~
src/instance.zig:1190:16: error: root struct of file 'mem' has no member named 'writeIntLittle'
        std.mem.writeIntLittle(T, &bytes, value);
        ~~~~~~~^~~~~~~~~~~~~~~
src/instance.zig:1190:16: error: root struct of file 'mem' has no member named 'writeIntLittle'
        std.mem.writeIntLittle(T, &bytes, value);
        ~~~~~~~^~~~~~~~~~~~~~~
src/instance.zig:1190:16: error: root struct of file 'mem' has no member named 'writeIntLittle'
        std.mem.writeIntLittle(T, &bytes, value);
        ~~~~~~~^~~~~~~~~~~~~~~
src/instance.zig:1190:16: error: root struct of file 'mem' has no member named 'writeIntLittle'
        std.mem.writeIntLittle(T, &bytes, value);
        ~~~~~~~^~~~~~~~~~~~~~~

[martinfouilleul]

When bundling the app, pass --wasm-backend bytebox

Isn't the backend already determined by the runtime version you're bundling with? (eg what happens if you build the runtime with --wasm-backend bytebox but then bundle without that option?)

[martinfouilleul]

Minor nitpick but I would also prefer if the inclusions of "backend_bytebox.c" / "backend_wasm3.c" where guarded by a macro inside runtime.c (rather than inside each file) and that macro was explicitly passed when building. This way it makes it clear we're only ever selecting a single backend, which is selected at build time.

[rdunnington]

@martinfouilleul

• Re compile errors: Which version of Zig are you using? Bytebox builds with 0.11.x, it's not compatible with 0.10 and I haven't tried with 0.12 yet.
• Re bundle.c: I think at the time I was not sure the best way to get the bundle tool to know about whether the runtime was using bytebox or not. If there was a mismatch there would be linker errors on windows. But I think I can make sure to pass down the macro when building to tool so it's baked in. I'll address that in an update to this PR.
• Re: I can update the backend selection stuff as well, no problem.

[martinfouilleul]

Re bundle.c: I think at the time I was not sure the best way to get the bundle tool to know about whether the runtime was using bytebox or not. If there was a mismatch there would be linker errors on windows. But I think I can make sure to pass down the macro when building to tool so it's baked in. I'll address that in an update to this PR.

Does the tool really need to know which backend the runtime was compiled with? If the runtime is statically linked to the wasm backend, the tool should just need to copy the runtime without knowing about the backend, right?

Re compile errors: Which version of Zig are you using?

Ah, I have 0.12.0-dev.1642+5f8641401. I'll try to get it working with 0.11

[martinfouilleul]

Ok, making progress, I got Zig 0.11 and it's compiling but there's still 2 issues:

• You're copying src/ext/bytebox/zig-out/lib/bytebox.lib, but on macOS it should be src/ext/bytebox/zig-out/lib/libbytebox.a
• When compiling with --release zig complains about unrecognized argument -. It seems that -Doptimize=ReleaseSafe is passed as separate letters...

[rdunnington]

Ok thanks! I think I forgot to test macOS so this makes sense. I'll take a look this week.

[rdunnington]

@martinfouilleul:

Does the tool really need to know which backend the runtime was compiled with? If the runtime is statically linked to the wasm backend, the tool should just need to copy the runtime without knowing about the backend, right?

Yeah that is true, if we statically linked ahead of time it would simplify the logic in bundle.c. The existing code copies wasm3.lib for bundle to link it, so I was mostly just following that pattern. I can probably update the dev script to statically link either wasm3 or bytebox depending on the selected backend and simplify the linking code in bundle.c appropriately.

[rdunnington]

There is one more detail to be aware of - due to the bundle program linking the libraries together on windows, it has to supply some special args to ensure bytebox links and runs correctly. Specifically:

• Some zig APIs require some ntdll libs
• Some zig APIs expect a large (8MB) stack, so we need to increase the default stack size in this case I was relying on the runtime flag to switch on this, but will plan on switching to a compile-time #define to make it more consistent with the planned linking behavior, unless you think otherwise.

[martinfouilleul]

Ah gotcha, I overlooked the fact we're linking at bundle-time on Windows, because this way we can use the linker to set the executable icon... We can temporarily rely on switches (if they only affect our own experiments with bytebox) or defines, but ideally we would avoid the root problem and statically link at build-time. I don't know how resources are stored in an windows executable, but I imagine they get a dedicated section in the exe file, how hard would it be to append the icon ourselves to a linked executable at bundle-time?

[martinfouilleul]

Btw this also means we still rely on MSVC being installed, and the bundle script being run in a visual studio command prompt, for the icon thing, which isn't ideal...

[rdunnington]

Yeah I agree requiring the MSVC linker isn't the greatest. But short of that, I see 2 options:

1. Bundle ResourceHacker.exe (or other icon-hacking software) with the orca install and use it to add an icon file after the fact. I'm not a huge fan of adding a non-open source .exe to the bundle step though.
2. Invest in understanding the .exe file format, where the icon is stored, what format it needs to be, and write the code to embed it ourselves. Based on some cursory searches, I haven't been able to find any existing open source implementations of this so it might be tricky.

[rdunnington]

Actually after a bit more searching I did find some decent reference code for a program called rcedit: https://github.com/electron/rcedit/blob/main/src/rescle.cc#L619 I tested the program on a scratch .exe and it works. :) Given this, I think I can just write the code to embed the icon, and avoid the extra linking step in bundle.c, which will let us link everything up-front ahead of time. bundle.c will still need to know about the selected backend, but I think we can do that with a compile time flag.

[martinfouilleul]

Given this, I think I can just write the code to embed the icon, and avoid the extra linking step in bundle.c, which will let us link everything up-front ahead of time. bundle.c will still need to know about the selected backend, but I think we can do that with a compile time flag.

Awesome, thanks!

[martinfouilleul]

Looks great! Last minute details:

• It seems --wasm-backend flag isn't used anymore in the CLI tool, since everything is linked statically beforehand.
• When building the runtime, you could avoid rebuilding wasm3 when the selected backend is bytebox

[rdunnington]

Oh yeah those are good callouts, I forgot to clean those up on this last pass. Thanks :)