Zig SDK Implementation

[elewis787]

Zig has added support for WASI and Freestanding wasm binaries. Given this support, zig seems like a good language to implement the spin SDK in.

Additionally, zig has first class support for interfacing with c libraries. Implementing a zig spin sdk should be similar to the tinygo sdk implementation that is using the wit-bindgen generated c files.

[elewis787]

I am interesting in implemented this. I am very new to zig but have started playing around with basic layouts here https://github.com/elewis787/spin-zig ( Heavy WIP currently ). First pass will focus on http support.

[elewis787]

Making decent progress on this. I am able to construct the HTTP handler flow and call the function using spin (simple test to set response header/status) . I am now working on translating the spin C http objects to zig. I am working on this in my spare time so I don't have a hard timeline but I am hoping to get this finalized over the next few days. .

[elewis787]

quick update - working through a few more items for the http components and translations but here is a quick sample/sneak peak that is at least in a POC state.

const std = @import("std");
const spin = @import("spin");

fn hello(req: *spin.Request, rw: *spin.ResponseWriter) void {
    _ = req;
    const n = rw.write("{\"foo\":\"bar\", \"baz\":\"boo\"}");
    if (@TypeOf(n) != usize ) {
        rw.status(std.http.Status.internal_server_error);
    }
    rw.status(std.http.Status.ok);
}

pub fn main() void {
    spin.Http.handle(hello);
}

I've been playing around with a few zig http libs to see if I could avoid some of the underlying socket reliance ( most use net.address - which is easy to replace in objects ). However, for now I have landed on writing a simple http package ... it will not be fully featured but it should cover the basics. Main reason for this was to be able to use some of the std http packages and to avoid some of the overlaps with the http server components. I also think this will set it up to be easily replaced as the std http package gets built out in zig.

More or less my focus is HTTP objects and parsing HTTP requests. I plan on using https://github.com/MasterQ32/zig-uri for the uri parsing ... but I will still need to do some request parsing ( headers and version etc.. )

The wasm bin looks as expected

( highlighting the exports below)

  (export "memory" (memory 0))
  (export "_start" (func $_start.command_export))
  (export "main" (func $main.command_export))
  (export "canonical_abi_realloc" (func $canonical_abi_realloc.command_export))
  (export "canonical_abi_free" (func $canonical_abi_free.command_export))
  (export "handle-http-request" (func $__wasm_export_spin_http_handle_http_request.command_export))
  (export "spin_http_handle_http_request" (func $spin_http_handle_http_request.command_export))

tracing through spin_http_handle_http_request shows how our handler eventually gets called...fun stuff 😎

[elewis787]

Quick update - running into an issue when setting headers on the response object.

Trace output

2022-08-01T04:22:18.931277Z  INFO spin_http_engine: Processing request for application spin-hello-zig on URI http://127.0.0.1:3000/hello
2022-08-01T04:22:18.931301Z TRACE spin_http_engine::spin: Executing request using the Spin executor for component zig-hello
2022-08-01T04:22:18.931334Z TRACE prepare_component{component="zig-hello" env=None args=None}: spin_engine: Preparing component zig-hello
2022-08-01T04:22:18.931344Z TRACE prepare_component{component="zig-hello" env=None args=None}: spin_engine: Creating store.
2022-08-01T04:22:18.933482Z TRACE spin_engine: Saving logs to "/Users/elewis/.spin/spin-hello-zig/logs/zig-hello_stdout.txt" "/Users/elewis/.spin/spin-hello-zig/logs/zig-hello_stderr.txt"
2022-08-01T04:22:18.933797Z ERROR spin_http_engine: Error processing request: wasm trap: out of bounds memory access
wasm backtrace:
    0: 0x28cf - <unknown>!dlfree
    1: 0x27f9 - <unknown>!free
    2: 0x7b77 - canonical_abi_free
                    at /Users/elewis/Projects/zig/github.com/elewis787/spin-zig/src/spin-http.c:28:3
    3: 0x4074a - <unknown>!canonical_abi_free.command_export

I am still working on identifying what is causing the out of bounds error. My potentially naive thought is this could be a pointer alignment issue between zig/c but I haven't been able to prove that.

Here are some thoughts/comments

• By adding debug messages to the c bindings, I have been able to follow the allocation of my header objects. Pointer addresses, values and alignment seem to matchup.
• This is not an issue when working with the response body ( which is why I am thinking a struct pointer alignment vs native u8 alignment )
• If there is not an alignment issue - then I could be missing a call to manually free mem and might be getting a bad memory value.
• The tinygo wasm output for free is a nop call - commenting out free in the c binds allows for the headers to be set in the response and results in a "successful" execution.

Some of the slowness here is drilling in deeper to zig ( really like it so far ).

At any rate - if anyone has advice let me know! I will likely move on to other areas and allow for the free to be a nop and circle back to this.

[itowlson]

Is it worth sharing what you have, either as a link to your branch or as a draft PR, in case anyone else wants to try to reproduce/help understand? (Not trying to press you, just a thought.)

[elewis787]

Is it worth sharing what you have, either as a link to your branch or as a draft PR, in case anyone else wants to try to reproduce/help understand? (Not trying to press you, just a thought.)

Absolutely - the branch I am working off is - https://github.com/elewis787/spin-zig/tree/elewis-http-impl

I will push a quick readme for building / running the example.

[radu-matei]

This is looking great, @elewis787, thanks for working on this!

[elewis787]

Thanks and no problem @radu-matei! Been a little slow going lately with workload but I am still working on it! More to come.

[HeCorr]

@elewis787 Hello, any progress on the SDK?

[elewis787]

Hey @HeCorr, unfortunately, I haven't. Here is where I got https://github.com/elewis787/spin-zig/tree/elewis-http-impl. I have been meaning to pick this backup but haven't found time to circle back just yet.