Improve Packaging of Edgee WASM Component

SachaMorard commented 4 days ago

Description

Currently, to build an Edgee WASM component, we need to download wasi_snapshot_preview1.reactor.wasm manually.

Investigate the current build process and identify potential optimizations.

Explore the possibility of adding wasi_snapshot_preview1.reactor.wasm as a direct dependency in Cargo.toml ?

Implement any feasible optimizations to streamline the build process.

Additional Context

This improvement will help streamline the build process, making it more efficient and user-friendly, especially to be prepared for Edgee component registry.

References

Our components repos are:

flyaruu commented 4 days ago

Ok, I have played a bit with the amplitude-component. If I use wasm32-wasip2 as target, some slight Cargo.toml wrangling, and Rust nightly, it just works without adaptation.

flyaruu commented 4 days ago

I'm not sure what our stance is on Rust nightly? For me nightly seems to work better than stable, at least for WASM.

CLEMENTINATOR commented 3 days ago

Ok, I have played a bit with the amplitude-component. If I use wasm32-wasip2 as target, some slight Cargo.toml wrangling, and Rust nightly, it just works without adaptation.

Can you describe the changes needed in Cargo.toml ? Otherwise, regarding nightly, (and wasip2), I guess as long as stuff doesn't break, lets go for it!

Jer1605 commented 3 days ago

Question @SachaMorard

Why was the decision made initially to download wasi_snapshot_preview1.reactor.wasm manually instead of including it directly as a dependency in Cargo.toml?

Was there a specific limitation or compatibility issue that led to this approach? Understanding the original reasoning might help us avoid potential pitfalls.

KokaKiwi commented 3 days ago

I would suggest taking a loot at wasmtime's wasi-preview1-component-adapter-provider crate, we could integrate that into the build process (like using Rust build.rs "build script"):

// build.rs
use wasi_preview1_component_adapter_provider::WASI_SNAPSHOT_PREVIEW1_REACTOR_ADAPTER;

fn main() {
    std::fs::write(
        "wasi_snapshot_preview1.reactor.wasm",
        WASI_SNAPSHOT_PREVIEW1_REACTOR_ADAPTER
    ).expect("failed to write bytes to file");
}

And maybe later get the whole component building into a dedicated tool (and integrate later into the future edgee CLI?), so we wouldn't have to ask component developers to "manually" write the wasm-tools command

SachaMorard commented 3 days ago

Question @SachaMorard

Why was the decision made initially to download wasi_snapshot_preview1.reactor.wasm manually instead of including it directly as a dependency in Cargo.toml?

Was there a specific limitation or compatibility issue that led to this approach? Understanding the original reasoning might help us avoid potential pitfalls.

Unfortunately, I don't know. I'm not the one who built the first component ;)

SachaMorard commented 3 days ago

I would suggest taking a loot at wasmtime's wasi-preview1-component-adapter-provider crate, we could integrate that into the build process (like using Rust build.rs "build script"):
// build.rs
use wasi_preview1_component_adapter_provider::WASI_SNAPSHOT_PREVIEW1_REACTOR_ADAPTER;

fn main() {
    std::fs::write(
        "wasi_snapshot_preview1.reactor.wasm",
        WASI_SNAPSHOT_PREVIEW1_REACTOR_ADAPTER
    ).expect("failed to write bytes to file");
}
And maybe later get the whole component building into a dedicated tool (and integrate later into the future edgee CLI?), so we wouldn't have to ask component developers to "manually" write the wasm-tools command

Ok for the rust components, but what about the js/python/go/... components?

SachaMorard commented 3 days ago

I'm not sure what our stance is on Rust nightly? For me nightly seems to work better than stable, at least for WASM.

I assume that using Rust nightly to build a component requires the same Rust version for the proxy, right?

flyaruu commented 3 days ago

I don't think so, the component doesn't even need to be based on Rust at all.

SachaMorard commented 3 days ago

Even with something like ./edgee component build ? ;)

KokaKiwi commented 3 days ago

The Rust version used to build a Rust-based component is completely unrelated to the Rust version used to build Edgee CLI even when using edgee component build since there's no interaction at ABI-level between the two (or not even Rust binary code at all in general if using JS or Golang-based component), the only interfacing done is at the proxy with WASM Component which has its own ABI, so you could build a Rust component using Rust nightly and another one with Rust 1.80.0 if you really want to and it would still works :)

alexcasalboni commented 3 days ago

From a developer experience (DX) point of view, the ideal would be NOT having to install Rust locally to build and run a new component in Go, JS, Python, etc.

Ideally, you just brew install edgee and then edgee component create/build/run without having to worry about Rust or wasi_snapshot_preview1.reactor.wasm.

Would it be possible to include the .wasm file directly in the edgee binary? it's only 54KB, right?

flyaruu commented 3 days ago

So some updates:

The issue with the url crate (requiring 'default-features=false', is also resolved by using the latest version (2.5.3 instead of 2.5.2), so it looks like it was a bug in url
I think the issue with the stable compiler (or rather the linker) is also a bug which has been fixed in nightly. It is also fixed in the beta channel, which, if nothing weird happens, will be stable next Thursday.

So in conclusion: I think the discussion about nightly will be irrelevant very soon.

KokaKiwi commented 3 days ago

We could use the crate i mentioned earlier in the edgee crate to include the .wasm file into the final executable

edgee-cloud / edgee