search

nginx / unit

NGINX Unit - universal web app server - a lightweight and versatile open source server that simplifies the application stack by natively executing application code across eight different programming language runtimes.
https://unit.nginx.org
Apache License 2.0
5.4k stars 331 forks source link

bump wasmtime to v24 #1406

Closed avahahn closed 2 months ago

avahahn commented 2 months ago

This PR bumps the libwasmtime version used in both the wasm and wasm-wasi-component modules to version 24.

This PR adds additions to the tooling to properly build libwasmtime for the wasm module. This introduces the need for cmake on the host. Dockerfiles are not updated to reflect this as part of this PR.

Fixes: https://github.com/nginx/unit/issues/1392

thresheek commented 2 months ago

Dockerfiles & packaging need to be adapted for this change, indeed.

ac000 commented 2 months ago

There's a little too much going on in this commit.

Can you extract the wasm-wasi-component changes into their own commit please. That's the important thing to get merged.

avahahn commented 2 months ago

This now only contains the wasm-wasi-component changes

ac000 commented 2 months ago

Testing it out...

Hmm, where to start!

So firstly I tried some of my C based wasi-http 0.2.0 components with this updated module but that resulted in failure, one using the reactor adaptor wouldn't load, the other using the proxy adaptor loaded, but errors out on requests.

Even after updating all the things; wit-bindgen, wasm-tools, wasi-http, same result... OK lets forget that for now. There are so many moving parts and everything seems to be in a continual state of flux...

So moving onto rust.

I can build and run our rust test application as described here on the updated module, so that;'s good!, but it's about the only thing I can run... (the pytest does also pass).

Trying the application from here (after fixing compiler issues, here it is for convenience)

use wasi::http::types::{
    Fields, IncomingRequest, OutgoingBody, OutgoingResponse, ResponseOutparam,
};

wasi::http::proxy::export!(Component);
use wasi::cli::environment::get_environment;

struct Component;

const HTML_BODY: &str = r#"<html>
    <head>
        <title>Hello from WebAssembly!</title>
    </head>
    <body>
      {{ENV_VARS}}
    </body>
</html>"#;

impl wasi::exports::http::incoming_handler::Guest for Component {
    fn handle(_request: IncomingRequest, response_out: ResponseOutparam) {
        let hdrs = Fields::new();
        let variables = format!("{:?}", get_environment());
        let mesg = String::from(HTML_BODY).replace("{{ENV_VARS}}", &variables);

        let _try = hdrs.set(
            &"Content-Type".to_string(),
            &[b"text/html; charset=utf-8".to_vec()],
        );
        let _try = hdrs.set(
            &"Content-Length".to_string(),
            &[mesg.len().to_string().as_bytes().to_vec()],
        );

        let resp = OutgoingResponse::new(hdrs);
        resp.set_status_code(200).unwrap();

        let body = resp.body().unwrap();
        ResponseOutparam::set(response_out, Ok(resp));

        let out = body.write().unwrap();
        out.blocking_write_and_flush(mesg.as_bytes()).unwrap();
        drop(out);

        OutgoingBody::finish(body, None).unwrap();
    }
}

Results in (when trying to load the component)

2024/08/29 00:37:16 [alert] 180726#180726 failed to create initial state

Caused by:
    0: failed to pre-instantiate the provided component
    1: component imports instance `wasi:cli/environment@0.2.1`, but a matching implementation was not found in the linker
    2: instance export `get-environment` has the wrong type
    3: function implementation is missing

I wonder if that has anything to do with changes like

-        wasmtime_wasi::preview2::command::add_to_linker(&mut linker)?;
-        wasmtime_wasi_http::proxy::add_only_http_to_linker(&mut linker)?;
+        add_to_linker_async(&mut linker)
+            .context("failed to add to linker")?;
+        //add_only_http_to_linker_sync(&mut linker)
+        //    .context("failed to add http to linker")?;

and perhaps

-            component,
+            component: proxy,

That prevent reactor (as opposed to proxy) component types from working?

-        wasmtime_wasi::preview2::command::add_to_linker(&mut linker)?;

That line in particular was added by

commit 60eb6c43a71fb0f6cf21759917e0247bdb88b892
Author: Andrew Clayton <a.clayton@nginx.com>
Date:   Tue Feb 13 16:21:49 2024 +0000

    Wasm-wc: Allow to use the 'reactor' adaptor again

    With the initial port to wasmtime 17 we could no longer use the
    'reactor' adaptor but had to switch to the more restrictive 'proxy'
    adaptor.

    This meant amongst other things (probably) we could no longer access the
    filesystem.

    Thanks to Joel Dice for pointing out the fix.

    With this we can go back to using the 'reactor' adaptor again and things
    are back to working as before.

    It's worth noting that you can use either the 'proxy' or 'reactor'
    adaptor depending on your requirements.

    Cc: Joel Dice <joel.dice@fermyon.com>
    Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

diff --git ./src/wasm-wasi-component/src/lib.rs ./src/wasm-wasi-component/src/lib.rs
index 888074ab..3ee40c4f 100644
--- ./src/wasm-wasi-component/src/lib.rs
+++ ./src/wasm-wasi-component/src/lib.rs
@@ -200,7 +200,8 @@ impl GlobalState {
         let component = Component::from_file(&engine, &global_config.component)
             .context("failed to compile component")?;
         let mut linker = Linker::<StoreState>::new(&engine);
-        wasmtime_wasi_http::proxy::add_to_linker(&mut linker)?;
+        wasmtime_wasi::preview2::command::add_to_linker(&mut linker)?;
+        wasmtime_wasi_http::proxy::add_only_http_to_linker(&mut linker)?;
         let component = linker
             .instantiate_pre(&component)
             .context("failed to pre-instantiate the provided component")?;
callahad commented 2 months ago

Will want to verify (and add automated tests?) for filesystem access, if we're not already testing.

ac000 commented 2 months ago

Hmm, so, the two add_to_linker's do allow the component to load and run, thanks!

However when then trying to get the environment variables (with cx.inherit_env();) things go awry...

The component loads, but then crashes when making a request...

$ curl localhost:8080/
curl: (18) transfer closed with 5014 bytes remaining to read
thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0xe93f - wit-component:shim!indirect-wasi:io/streams@0.2.1-[method]output-stream.blocking-write-and-flush
           1: 0x3471 - <unknown>!<wasm function 43>
    1: Buffer too large for blocking-write-and-flush (expected at most 4096)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024/08/30 14:22:30 [alert] 431991#431991 app process 431992 exited on signal 6 (core dumped)
thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0xe93f - wit-component:shim!indirect-wasi:io/streams@0.2.1-[method]output-stream.blocking-write-and-flush
           1: 0x3471 - <unknown>!<wasm function 43>
    1: Buffer too large for blocking-write-and-flush (expected at most 4096)

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: wasm_wasi_component::GlobalState::run::{{closure}}::{{closure}}
   4: std::panicking::try

In the meantime I'll try and see if filesystem access is working...

javorszky commented 2 months ago

@ac000 do you have the code for your C based wasi component handy so I can reproduce the issue you're seeing?

callahad commented 2 months ago

That's a strange crash. I am able to get the example module from above to work and print out envvars in its response, so I'm not sure where / how it's failing on your end.

I can't get filesystem access to work, neither in Unit nor with wasmtime serve --dir PATH. So something's weird there and needs deeper investigation.

ac000 commented 2 months ago

@javorszky

There is https://git.digital-domain.net/project_blackbird.git/tree/c/wasi-http/0.2.0/echo-request for example. But it's giving me the same issue as the rust application in my initial comment.

ac000 commented 2 months ago

Well, not exactly the same error...

thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0xe8f7 - wit-component:shim!indirect-wasi:http/types@0.2.0-[static]response-outparam.set
           1: 0x473d - module.wasm!wasi_http_types_static_response_outparam_set
           2: 0x59f6 - module.wasm!exports_wasi_http_incoming_handler_handle
           3: 0x4c87 - module.wasm!__wasm_export_exports_wasi_http_incoming_handler_handle
    1: unknown handle index 0

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: wasm_wasi_component::GlobalState::run::{{closure}}::{{closure}}
   4: std::panicking::try
ac000 commented 2 months ago

That's a strange crash. I am able to get the example module from above to work and print out envvars in its response, so I'm not sure where / how it's failing on your end.

I can't get filesystem access to work, neither in Unit nor with wasmtime serve --dir PATH. So something's weird there and needs deeper investigation.

Seeing the same thing

thread '<unnamed>' panicked at src/lib.rs:27:51:
Unable to write file: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/tmp/wasm-wc-fs-test\" could be opened" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x8085 - et.wasm!__rust_start_panic
           1: 0x7e50 - et.wasm!rust_panic
           2: 0x7c06 - et.wasm!_ZN3std9panicking20rust_panic_with_hook17haaccccfe5a373acdE
           3: 0x70ad - et.wasm!_ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17hea3feb189860033dE
           4: 0x7014 - et.wasm!_ZN3std10sys_common9backtrace26__rust_end_short_backtrace17hd9f533f0d74942c6E
           5: 0x779e - et.wasm!rust_begin_unwind
           6: 0xac53 - et.wasm!_ZN4core9panicking9panic_fmt17hcba33b52fb7a6f32E
           7: 0xc881 - et.wasm!_ZN4core6result13unwrap_failed17he1e30e4458e0844aE
           8: 0x2272 - et.wasm!_ZN91_$LT$et..Component$u20$as$u20$wasi..proxy..exports..wasi..http..incoming_handler..Guest$GT$6handle17h1e747c577217c4a8E
           9: 0x2315 - et.wasm!wasi:http/incoming-handler@0.2.1#handle
    1: wasm trap: wasm `unreachable` instruction executed

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: wasm_wasi_component::GlobalState::run::{{closure}}::{{closure}}
   4: std::panicking::try

I wonder if it's due to the removal of

                 for dir in self.global_config.dirs.iter() {
-                    let fd = Dir::open_ambient_dir(dir, ambient_authority())
-                        .with_context(|| {
-                            format!("failed to open directory '{dir}'")
-                        })?;
ac000 commented 2 months ago

The other slightly strange thing is that the preopened_dir function now seems to only take three arguments whereas we're passing in four...

ac000 commented 2 months ago

As an aside, with most things wasm/rust seemingly broken for me, I double checked that the wasm language module and unit-wasm examples still work with libwasmtime v24, including filesystem access, so that's something!

ac000 commented 2 months ago

Having setup a clean rust/cargo environment (using rustup rather than distro packages) I can now load and run the above rust component and get the environment variables.

But filesystem access is still not working. The language module does open the directory at request time

[pid 17597] openat(AT_FDCWD, "/var/tmp", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 17

but I guess it doesn't set the ambient authority on it like it used to...

javorszky commented 2 months ago

The other slightly strange thing is that the preopened_dir function now seems to only take three arguments whereas we're passing in four...

I think we're using the method on the WasiCtxBuilder, which does take 2 dirs: https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/struct.WasiCtxBuilder.html#method.preopened_dir

ac000 commented 2 months ago

One thing I notice is that with a component that works under wasmtime 17, we have these filesystems imports

  import wasi:filesystem/types@0.2.0;
  import wasi:filesystem/preopens@0.2.0;

Which seem to be missing in the latest component built with cargo-component v0.15.0

ac000 commented 2 months ago

Building the Rust code from https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-tutorial.md as a command component (i.e. src/main.rs with a main() function) does include them. That then runs under wasmtime run and works...

ac000 commented 2 months ago

Tried about a bazilliion different things, getting nowhere fast. Even tried taking cargo-component out the equation by using a mixture of cargo-build and wasm-tools but still crashes

thread '<unnamed>' panicked at src/lib.rs:27:51:
Unable to write file: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/tmp/wasm-wc-fs-test\" could be opened" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x207fa - et.wasm!__rust_start_panic
           1: 0x205c5 - et.wasm!rust_panic
           2: 0x2037b - et.wasm!_ZN3std9panicking20rust_panic_with_hook17h8bb4f94cf9e68e20E
           3: 0x1f79d - et.wasm!_ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17h2c354a3372fcca00E
           4: 0x1f704 - et.wasm!_ZN3std10sys_common9backtrace26__rust_end_short_backtrace17hf791a0fba63af5e9E
           5: 0x1ff13 - et.wasm!rust_begin_unwind
           6: 0x25513 - et.wasm!_ZN4core9panicking9panic_fmt17haadcabb094f4c85bE
           7: 0x27186 - et.wasm!_ZN4core6result13unwrap_failed17hd09ff05c9c2fb25fE
           8: 0x357b - et.wasm!_ZN4core6result19Result$LT$T$C$E$GT$6expect17h128bef27c8ee3bccE
           9: 0x1f07 - et.wasm!_ZN91_$LT$et..Component$u20$as$u20$wasi..proxy..exports..wasi..http..incoming_handler..Guest$GT$6handle17h08a6114a11b81d6cE
          10: 0x7ff8 - et.wasm!_ZN4wasi5proxy7exports4wasi4http16incoming_handler19_export_handle_cabi17hdc42bed91e5ceefbE
          11: 0x2919 - et.wasm!wasi:http/incoming-handler@0.2.1#handle
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    1: wasm trap: wasm `unreachable` instruction executed

Gotta love the symbol mangling, looks like something Rust picked up from C++

javorszky commented 2 months ago

Opened issue on the wasmtime repository: https://github.com/bytecodealliance/wasmtime/issues/9194

ac000 commented 2 months ago

OK, so removing proxy = true from Cargo.toml does make it work on my Fedora VM (with rust/cargo from rustup).

Things are still totally b0rked on my Fedora workstation using distribution packages of rust/cargo.

thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x108ac - wit-component:shim!indirect-wasi:io/streams@0.2.1-[method]output-stream.blocking-write-and-flush
           1: 0x3aa4 - <unknown>!<wasm function 43>
    1: Buffer too large for blocking-write-and-flush (expected at most 4096)

That is not even trying any filesystem stuff...

ac000 commented 2 months ago

If we're going ahead with this, then those commits should be squashed and have a commit subject prefix of wasm-wc...

I'd suggest something like

wasm-wc: Update the wasmtime-crate to v24.0.0
ac000 commented 2 months ago

OK, quick thing...

Could you run from the repository root on your ava-wasmtime branch

$ rustfmt --edition 2021 src/wasm-wasi-component/src/lib.rs
$ git add src/wasm-wasi-component/src/lib.rs
$ git commit --amend --no-edit
$ git push -f ... ...
ac000 commented 2 months ago

NOTE: NOT A BUG

I had

fs::write("/var/tmp/wasm-wc-fs-test", "test").expect("Unable to write file");

not realising .expect() would cause a panic if the call failed!

Simply changing it to

let _ = fs::write("/var/tmp/wasm-wc-fs-test", "test");

gets rid of the crash...

I wasn't seeing the crash with the previous version of the module as a I was using C for the components (really need to get this working again with this version of the module) and was not manually triggering a crash if the open(2) failed...

================================================================================================= Just for the record. This does have the known issue in that if you try to access a directory that isn't pre-opened, the language module will crash (this doesn't happen with the current module)

2024/09/04 21:30:49 [info] 4111#4111 "my-wasm-component" application started
thread '<unnamed>' panicked at src/lib.rs:28:55:
Unable to write file: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/var/tmp/wasm-wc-fs-test\" could be opened" }
stack backtrace:
thread 'tokio-runtime-worker' panicked at src/lib.rs:248:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x207fa - et.wasm!__rust_start_panic
           1: 0x205c5 - et.wasm!rust_panic
           2: 0x2037b - et.wasm!_ZN3std9panicking20rust_panic_with_hook17h8bb4f94cf9e68e20E
           3: 0x1f79d - et.wasm!_ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17h2c354a3372fcca00E
           4: 0x1f704 - et.wasm!_ZN3std10sys_common9backtrace26__rust_end_short_backtrace17hf791a0fba63af5e9E
           5: 0x1ff13 - et.wasm!rust_begin_unwind
           6: 0x25513 - et.wasm!_ZN4core9panicking9panic_fmt17haadcabb094f4c85bE
           7: 0x27186 - et.wasm!_ZN4core6result13unwrap_failed17hd09ff05c9c2fb25fE
           8: 0x357b - et.wasm!_ZN4core6result19Result$LT$T$C$E$GT$6expect17h128bef27c8ee3bccE
           9: 0x1f07 - et.wasm!_ZN91_$LT$et..Component$u20$as$u20$wasi..proxy..exports..wasi..http..incoming_handler..Guest$GT$6handle17h08a6114a11b81d6cE
          10: 0x7ff8 - et.wasm!_ZN4wasi5proxy7exports4wasi4http16incoming_handler19_export_handle_cabi17hdc42bed91e5ceefbE
          11: 0x2919 - et.wasm!wasi:http/incoming-handler@0.2.1#handle
    1: wasm trap: wasm `unreachable` instruction executed

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
stack backtrace:
   0:     0x7fdf149ee4f5 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd736fd5964392270
   1:     0x7fdf14a13f1b - core::fmt::write::hc6043626647b98ea
   2:     0x7fdf149eb05f - std::io::Write::write_fmt::h0d24b3e0473045db
   3:     0x7fdf149ee2ce - std::sys_common::backtrace::print::h45eb8174d25a1e76
   4:     0x7fdf149ef4a9 - std::panicking::default_hook::{{closure}}::haf3f0170eb4f3b53
   5:     0x7fdf149ef24a - std::panicking::default_hook::hb5d3b27aa9f6dcda
   6:     0x7fdf149ef943 - std::panicking::rust_panic_with_hook::h6b49d59f86ee588c
   7:     0x7fdf149ef824 - std::panicking::begin_panic_handler::{{closure}}::hd4c2f7ed79b82b70
   8:     0x7fdf149ee9b9 - std::sys_common::backtrace::__rust_end_short_backtrace::h2946d6d32d7ea1ad
   9:     0x7fdf149ef557 - rust_begin_unwind
  10:     0x7fdf14301c43 - core::panicking::panic_fmt::ha02418e5cd774672
  11:     0x7fdf14302136 - core::result::unwrap_failed::h55f86ada3ace5ed2
  12:     0x7fdf14406623 - wasm_wasi_component::GlobalState::run::{{closure}}::{{closure}}::hde6d9ddccf4e74b6
  13:     0x7fdf1440e978 - std::panicking::try::hedbe02d01cfa8c6e
2024/09/04 21:30:52 [alert] 4110#4110 app process 4111 exited on signal 6 (core dumped)