acodervic commented 1 year ago

if this is possible,please give me a exmple code . thank you ! Zware is a amazing tool !

Note: i want to call zig from wasm and call wasm from zig .

acodervic commented 1 year ago

🙏

malcolmstill commented 1 year ago

@acodervic thank you for your intereset.

Have a look at https://github.com/malcolmstill/zware-doom/blob/master/src/interface.zig. This repo is embedding Doom compiled to wasm (from https://github.com/malcolmstill/doom) in a zig program that provides graphics and input.

The interface.zig file is generated by running zware-gen and passing it the doom.wasm file. It will dump out zig code that defines a const Api = struct that let's you call exported functions and will stub out functions that the .wasm imports, you will need to provide implementations of these functions (other than the standard wasi functions which are provided by zware.wasi.

You can see for example the following host function:

pub fn ZwareDoomRenderFrame(vm: *zware.VirtualMachine) zware.WasmError!void {
    const screen_len = vm.popOperand(u32);
    const screen_ptr = vm.popOperand(u32);

    const memory = try vm.inst.getMemory(0);
    const data = memory.memory();
    const screen = data[screen_ptr .. screen_ptr + screen_len];

    glfw.renderFrame(screen);

    try vm.pushOperand(u64, 0);
}

...we have access to the WebAssembly stack and memory through vm.. The function "receives" arguments by popping from the WebAssembly stack and "returns" values by pushing to the WebAssembly stack.

malcolmstill commented 1 year ago

The previous reply covered calling WebAssembly code (from zig) via the Api struct and WebAssembly calling zig code (registered via exposeHostFunction).

As to the issue title of compiling zig code to .wasm check out https://github.com/malcolmstill/DOOM/blob/master/build.zig. That's actually compiling C code to .wasm but with the zig build system. A couple of things to note:

I invoke the compilation of that build.zig with zig build -Dtarget=wasm32-wasi-musl -Doptimize=ReleaseFast. I.e. I explicitly set the target to wasm32-wasi-musl. This along with exe.linkLibC(); in build.zig will use musl backed by wasi "system calls".
The build.zig file also specifies exe.import_symbols = true;. What this does is make sure symbols that aren't found during linking are added to the .wasm imports section (so that we can provide them) rather than the linker erroring.

malcolmstill commented 1 year ago

Let me know if that helps (or indeed if it doesn't help!) and I can try to clarify. Feel free to share any code you have and I can try and help from that direction.

acodervic commented 1 year ago

Let me know if that helps (or indeed if it doesn't help!) and I can try to clarify. Feel free to share any code you have and I can try and help from that direction.

Ok,I will try it . Thanks for you help. If it is worked or not, I will report my result.

acodervic commented 1 year ago

Let me know if that helps (or indeed if it doesn't help!) and I can try to clarify. Feel free to share any code you have and I can try and help from that direction.

I get errors when i build doom to wasm file.

zig build 
/home/x/Downloads/chromeDownload/DOOM-master/build.zig:93:12: error: member function expected 1 argument(s), found 2
        exe.addCSourceFile(file, &flags);
        ~~~^~~~~~~~~~~~~~~
/home/x/.zvm/0.11.0/lib/std/Build/Step/Compile.zig:905:5: note: function declared here
pub fn addCSourceFile(self: *Compile, source: CSourceFile) void {
~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    runBuild__anon_7164: /home/w/.zvm/0.11.0/lib/std/Build.zig:1638:27
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

I just want to build a simple app (like a add function and return result (input:x,y output:z ) ) to wasm and call it with zware. Is there has any easy way to do it ?

acodervic commented 1 year ago

Forgive me, Build doom to wasm and running it with zware is amazing. But someone just need a simpile example to clarify how to do it .

malcolmstill commented 1 year ago

@acodervic right, so there was an issue with the build.zig in https://github.com/malcolmstill/doom. I've pushed a change that I think works for both zig master and zig 0.11 (the version of zig used to build the .wasm)

malcolmstill commented 1 year ago

Okay, so let's do this from scratch.

Step 1. Our WebAssembly program

We'll do this part in zig, but this could really be any language that emits wasm. We'll also lean here on wasi support.

Using the master version of zig, let's zig init-exe (it doesn't really matter what version of zig we use here in the same way it doesn't really matter our source language), in (say) a directory called program. We get the following main.zig (plus the standard `build.zig):

const std = @import("std");

pub fn main() !void {
    // Prints to stderr (it's a shortcut based on `std.io.getStdErr()`)
    std.debug.print("All your {s} are belong to us.\n", .{"codebase"});

    // stdout is for the actual output of your application, for example if you
    // are implementing gzip, then only the compressed bytes should be sent to
    // stdout, not any debugging messages.
    const stdout_file = std.io.getStdOut().writer();
    var bw = std.io.bufferedWriter(stdout_file);
    const stdout = bw.writer();

    try stdout.print("Run `zig build test` to run the tests.\n", .{});

    try bw.flush(); // don't forget to flush!
}

We don't need to make any changes to this. We can compile this to webassembly simply by running:

zig build -Dtarget=wasm32-wasi-musl -Doptimize=ReleaseFast

...and we should get a zig-out/bin/program.wasm

Step 2. Our zware program

Let's init another exe, but this time we need to be particular about zig version, let's say this in a directory called wrapper. In particular we need to use zig 0.11

[!NOTE]
zware needs some updates to work with the current zig master. I'd like to aim for zware supporting the latest stable release and master at the same time.

Having inited the new exe we're going to make the following changes:

Copy the program.wasm from the program directory to wrapper/src/program.wasm
Add a build.zig.zon to define the dependency on zware
Add a couple of lines to build.zig to make the zware module available to our program.
Create an interface.zig file based upon the particular .wasm file. We could do this by hand, but the zware-gen program can do this for us.
Update main.zig to embed program.wasm, initialise zware, initialise some stuff from interface.zig and call the _start function of program.wasm

Change 1 is easy.

Change 2:

Our build.zig.zon will look like:

.{
    .name = "wrapper",
    .version = "0.0.1",
    .paths = .{""},
    .dependencies = .{
        .zware = .{
            .url = "https://github.com/malcolmstill/zware/archive/be5ccf4d741f2fab37ad4174779d3d4f465f19f1.tar.gz",
            .hash = "1220173793052d32ae35beb0e5412b1f57bd3cf4c4b2db585adbb292f558e20105e2",
        },
    },
}

Change 3:

After the definition of const exe definition in build.zig we make the zware module available to our wrapper:

    const zware = b.dependency("zware", .{ .target = target, .optimize = optimize });
    exe.addModule("zware", zware.module("zware"));

Change 4:

The interface.zig will look like this:

const std = @import("std");
const zware = @import("zware");

pub fn initHostFunctions(store: *zware.Store) !void {
    try store.exposeHostFunction("wasi_snapshot_preview1", "proc_exit", zware.wasi.proc_exit, &[_]zware.ValType{.I32}, &[_]zware.ValType{});
    try store.exposeHostFunction("wasi_snapshot_preview1", "fd_write", zware.wasi.fd_write, &[_]zware.ValType{ .I32, .I32, .I32, .I32 }, &[_]zware.ValType{.I32});
}

pub const Api = struct {
    instance: *zware.Instance,

    const Self = @This();

    pub fn init(instance: *zware.Instance) Self {
        return .{ .instance = instance };
    }

    pub fn _start(self: *Self) !void {
        var in = [_]u64{};
        var out = [_]u64{};
        try self.instance.invoke("_start", in[0..], out[0..], .{});
    }
};

Again, you can write this by hand, but zware-gen will generate this file.

[!NOTE]
In the case of program.wasm we are only calling from zig into wasm, and so we don't need to make any changes to interface.zig. However, if we are calling from wasm into zig, zware-gen will only have created stub functions that you will have to implement.

Change 5:

Update the zig init-exe main.zig to look like this:

const std = @import("std");
const zware = @import("zware");
const Store = zware.Store;
const Module = zware.Module;
const Instance = zware.Instance;
const GeneralPurposeAllocator = std.heap.GeneralPurposeAllocator;
var gpa = GeneralPurposeAllocator(.{}){};
const interface = @import("interface.zig");

pub fn main() !void {
    defer _ = gpa.deinit();
    const alloc = gpa.allocator();

    const bytes = @embedFile("program.wasm");

    var store = Store.init(alloc);
    defer store.deinit();

    var module = Module.init(alloc, bytes);
    defer module.deinit();
    try module.decode();

    try interface.initHostFunctions(&store);

    var instance = Instance.init(alloc, &store, module);
    try instance.instantiate();
    defer instance.deinit();

    var api = interface.Api.init(&instance);

    try api._start();
}

We initialise zware. We run try interface.initHostFunctions(&store); to setup imports / exports between zig and the wasm module. We init interface.Api.init(&instance); and finally call try api._start().

This should output:

➜  wrapper git:(master) ✗ zig-out/bin/wrapper
All your codebase are belong to us.
Run `zig build test` to run the tests.

malcolmstill commented 1 year ago

@acodervic let me know if that clarifies things, or if there's any more detail I can give (which I am more than happy to do!)

acodervic commented 1 year ago

@acodervic let me know if that clarifies things, or if there's any more detail I can give (which I am more than happy to do!)

Please tell me how to export a add function to wasm and call it with zware .Thanks !

//main.zig
const std = @import("std");

pub fn main() !void {
    // Prints to stderr (it's a shortcut based on `std.io.getStdErr()`)
    std.debug.print("All your {s} are belong to us.\n", .{"codebase"});

    // stdout is for the actual output of your application, for example if you
    // are implementing gzip, then only the compressed bytes should be sent to
    // stdout, not any debugging messages.
    const stdout_file = std.io.getStdOut().writer();
    var bw = std.io.bufferedWriter(stdout_file);
    const stdout = bw.writer();

    try stdout.print("Run `zig build test` to run the tests.\n", .{});

    try bw.flush(); // don't forget to flush!
}
export fn add(a: i32, b: i32) i32 {
    return a + b;
}

test "simple test" {
    var list = std.ArrayList(i32).init(std.testing.allocator);
    defer list.deinit(); // try commenting this out and see if zig detects the memory leak!
    try list.append(42);
    try std.testing.expectEqual(@as(i32, 42), list.pop());
}

malcolmstill commented 1 year ago

Right, let's try starting our WebAssembly program with zig init-lib:

This gives us this main.zig (which we won't modify):

const std = @import("std");
const testing = std.testing;

export fn add(a: i32, b: i32) i32 {
    return a + b;
}

test "basic add functionality" {
    try testing.expect(add(3, 7) == 10);
}

Now the build.zig will contain the following:

  const lib = b.addStaticLibrary(.{
        .name = "libr",
        // In this case the main source file is merely a path, however, in more
        // complicated build scripts, this could be a generated file.
        .root_source_file = .{ .path = "src/main.zig" },
        .target = target,
        .optimize = optimize,
    });

It seems if we change addStaticLibrary to addSharedLibrary:

    const lib = b.addSharedLibrary(.{
        .name = "libr",
        // In this case the main source file is merely a path, however, in more
        // complicated build scripts, this could be a generated file.
        .root_source_file = .{ .path = "src/main.zig" },
        .target = target,
        .optimize = optimize,
    });

and build with:

zig build -Dtarget=wasm32-freestanding

[!IMPORTANT]
We're using wasm32-freestanding here rather than wasm32-wasi-musl as in the zig init-exe example

[!WARNING]
This seems to print the following warning: zig build-lib libr ReleaseFast wasm32-freestanding: error: warning(link): unexpected LLD stderr: wasm-ld: warning: creating shared libraries, with -shared, is not yet stable

So maybe this isn't the "correct" way to build this library, or maybe this is just the state of things. I don't know.

We will get a zig-out/lib/libr.wasm which seems to have the add function as an export:

libr git:(master) ✗ wasm-objdump -x zig-out/lib/libr.wasm 

libr.wasm:      file format wasm 0x1

Section Details:

Custom:
 - name: "dylink.0"
 - mem_size     : 0
 - mem_p2align  : 0
 - table_size   : 0
 - table_p2align: 0
Type[2]:
 - type[0] () -> nil
 - type[1] (i32, i32) -> i32
Import[3]:
 - memory[0] pages: initial=0 <- env.memory
 - global[0] i32 mutable=0 <- env.__memory_base
 - global[1] i32 mutable=0 <- env.__table_base
Function[3]:
 - func[0] sig=0 <__wasm_call_ctors>
 - func[1] sig=0 <__wasm_apply_data_relocs>
 - func[2] sig=1 <add>
Export[2]:
 - func[1] <__wasm_apply_data_relocs> -> "__wasm_apply_data_relocs"
 - func[2] <add> -> "add"
Code[3]:
 - func[0] size=2 <__wasm_call_ctors>
 - func[1] size=2 <__wasm_apply_data_relocs>
 - func[2] size=7 <add>
Custom:
 - name: ".debug_abbrev"
Custom:
 - name: ".debug_info"
Custom:
 - name: ".debug_str"
Custom:
 - name: ".debug_pubnames"
Custom:
 - name: ".debug_pubtypes"
Custom:
 - name: ".debug_line"
Custom:
 - name: "name"
 - func[0] <__wasm_call_ctors>
 - func[1] <__wasm_apply_data_relocs>
 - func[2] <add>
 - global[0] <__memory_base>
 - global[1] <__table_base>
Custom:
 - name: "producers"
Custom:
 - name: "target_features"
  - [+] mutable-globals
  - [+] sign-ext

[!NOTE]
It seems just adding the export fn add(... in the first example doesn't actually export that function when building a binary. Hence we're trying with zig init-lib

Running zware-gen against libr.wasm will give us:

const std = @import("std");
const zware = @import("zware");

pub fn initHostFunctions(store: *zware.Store) !void {
    _ = store;
}

pub const Api = struct {
    instance: *zware.Instance,

    const Self = @This();

    pub fn init(instance: *zware.Instance) Self {
        return .{ .instance = instance };
    }

    pub fn __wasm_apply_data_relocs(self: *Self) !void {
        var in = [_]u64{};
        var out = [_]u64{};
        try self.instance.invoke("__wasm_apply_data_relocs", in[0..], out[0..], .{});
    }

    pub fn add(self: *Self, param0: i32, param1: i32) !i32 {
        var in = [_]u64{ @bitCast(@as(i64, param0)), @bitCast(@as(i64, param1)) };
        var out = [_]u64{0};
        try self.instance.invoke("add", in[0..], out[0..], .{});
        return @bitCast(@as(u32, @truncate(out[0])));
    }
};

And so we can have a main.zig in our zware program that looks like:

const std = @import("std");
const zware = @import("zware");
const Store = zware.Store;
const Module = zware.Module;
const Instance = zware.Instance;
const GeneralPurposeAllocator = std.heap.GeneralPurposeAllocator;
var gpa = GeneralPurposeAllocator(.{}){};
const interface = @import("interface.zig");

pub fn main() !void {
    defer _ = gpa.deinit();
    const alloc = gpa.allocator();

    const bytes = @embedFile("libr.wasm");

    var store = Store.init(alloc);
    defer store.deinit();

    var module = Module.init(alloc, bytes);
    defer module.deinit();
    try module.decode();

    // In this case we don't have any host functions:
    // try interface.initHostFunctions(&store);

    var instance = Instance.init(alloc, &store, module);
    try instance.instantiate();
    defer instance.deinit();

    var api = interface.Api.init(&instance);

    const result = try api.add(56, -14);
    std.debug.print("result = {}\n", .{result});
}

...well actually, that's not quite right. The wrapper will compile and we can run it, but you'll get a error: ImportNotFound error. As you can see from the wasm-objdump (which is part of https://github.com/WebAssembly/wabt), as built the libr.wasm is expecting the host to provide the memory and a couple of globals.

Let's add the memory and globals manually:

const std = @import("std");
const zware = @import("zware");
const Store = zware.Store;
const Module = zware.Module;
const Instance = zware.Instance;
const GeneralPurposeAllocator = std.heap.GeneralPurposeAllocator;
var gpa = GeneralPurposeAllocator(.{}){};
const interface = @import("interface.zig");

pub fn main() !void {
    defer _ = gpa.deinit();
    const alloc = gpa.allocator();

    const bytes = @embedFile("libr.wasm");

    var store = Store.init(alloc);
    defer store.deinit();

    var module = Module.init(alloc, bytes);
    defer module.deinit();
    try module.decode();

    // In this case we don't have any host functions:
    // try interface.initHostFunctions(&store);

    const mem_handle = try store.addMemory(0, null);
    const mem_base_handle = try store.addGlobal(.{ .valtype = .I32, .mutability = .Immutable, .value = 0 });
    const table_base_handle = try store.addGlobal(.{ .valtype = .I32, .mutability = .Immutable, .value = 0 });
    try store.@"export"("env", "memory", .Mem, mem_handle);
    try store.@"export"("env", "__memory_base", .Global, mem_base_handle);
    try store.@"export"("env", "__table_base", .Global, table_base_handle);

    var instance = Instance.init(alloc, &store, module);
    try instance.instantiate();
    defer instance.deinit();

    var api = interface.Api.init(&instance);

    const result = try api.add(56, -14);
    std.debug.print("result = {}\n", .{result});
}

and finally:

➜  wrapper-libr git:(master) ✗ zig-out/bin/wrapper           
result = 42

malcolmstill commented 1 year ago

@acodervic so this library case is not ideal at the moment (having to manually define the memory).

There are a couple of things to improve here:

Maybe I'm not building the libr.wasm correctly and there are some other options that change how it's built so that it defines its own memory and therefore doesn't require the host to supply it.
If that's not possible, or maybe regardless, improve zware-gen so as to do some of the lifting here.

acodervic commented 1 year ago

I cant export "add" function in interface.zig by zware-gen.

test-wasm.zip

zig build -Dtarget=wasm32-freestanding
zware/tools/zware-gen$ zig build run  -- /home/xxx/test-wasm/zig-out/lib/libr.wasm 
const std = @import("std");
const zware = @import("zware");

pub fn initHostFunctions(store: *zware.Store) !void {
}

pub const Api = struct {
        instance: *zware.Instance,

        const Self = @This();

        pub fn init(instance: *zware.Instance) Self {
                return .{ .instance = instance };
        }

}

malcolmstill commented 12 months ago

@acodervic that looks like it should work. I pulled down your .zip and when I run zware-gen on the built .wasm I get:

const std = @import("std");
const zware = @import("zware");

pub fn initHostFunctions(store: *zware.Store) !void {
}

pub const Api = struct {
    instance: *zware.Instance,

    const Self = @This();

    pub fn init(instance: *zware.Instance) Self {
        return .{ .instance = instance };
    }

    pub fn __wasm_apply_data_relocs(self: *Self) !void {
        var in = [_]u64{};
        var out = [_]u64{};
        try self.instance.invoke("__wasm_apply_data_relocs", in[0..], out[0..], .{});
    }

    pub fn add(self: *Self, param0: i32, param1: i32) !i32 {
        var in = [_]u64{@bitCast(@as(i64, param0)), @bitCast(@as(i64, param1))};
        var out = [_]u64{0};
        try self.instance.invoke("add", in[0..], out[0..], .{});
        return @bitCast(@as(u32, @truncate(out[0])));
    }
};

malcolmstill / zware

How to compile zig code to wasm and run it with zware ? #218

Step 1. Our WebAssembly program

Step 2. Our zware program

Change 1 is easy.

Change 2:

Change 3:

Change 4:

Change 5: