Open JMLX42 opened 4 years ago
OK I might have missed a bit of context, specifically "why" I want this to happen. And I mean "why" as in personally/philosophically, not just technically.
So here it is.
I strongly believe the sum of all of humanity's code base would have an order of magnitude bigger impact if all the ingenious, incredible and talent-driven code bases that already exist could be easily leveraged by one another. Free from the input source code language, scalability matters and interoperability problems.
In other words: we already got some frakin' huge and capable software pieces. Now imagine what problems we could solve by putting those pieces together.
Application level Web APIs are good. But building and maintaining APIs require a lot of high level description, different formats, different discovery mechanisms, etc... And each software can have it's own API format, transport layer, and so on. Those APIs open a window of possibility, but with always a different shape, handle and span. They come in too many various sizes and shapes. Having RPC interoperability at the VM layer would solve a lot of that.
WASM goes toward that goal of portability. I think WASM RPC goes one step toward the goal of interoperability.
Additional notes: Ethereum WebAssembly (EWASM) is - among many things - a smart contract oriented version of RPC for WASM. I'm not saying EWASM should/would build smart contract function calls on top of the potential WASM RPC standard I'm talking about here.
I'm saying there is traction not just for distributed workloads, but potentially decentralized too.
Smart contracts and how their interface allow them to call each others are one of the biggest addition of Ethereum on top of Bitcoin. And at the core of a thriving ecosystem. My understanding is that they chose to build on top of WASM to build on top of the promise of portability and abstract the source code language. Interoperability being added by the blockchain smart contract driven function interfaces/calls built as an addition on top of WASM.
The WASM RPC proposal I'm addressing here is similar in a way. But those interfaces would not be ad-hoc to Ethereum. Instead, they would leverage the existing WASM, WAT and Interface Types proposals to achieve a similar goal of interoperability between machines/computing units.
This would seem more appropriate to suggest for WASI, if anything: https://github.com/WebAssembly/WASI/issues
Wasm itself is mostly a computational substrate, and by design doesn't contain any APIs in the traditional sense.
Related, maybe: https://github.com/WebAssembly/wasi-http-proxy
@aardappel thank you for your feedback!
Related, maybe: https://github.com/WebAssembly/wasi-http-proxy
The README doesn't tell much about the "http proxy" side of it. Can you please clarify?
This would seem more appropriate to suggest for WASI, if anything: https://github.com/WebAssembly/WASI/issues
IDK yet what is the full extend of the potential proposal. So I'm not sure what layer is the best one for now. Can't I/we start digging and see how deep the rabbit hole goes? I'm not sure we can just think our way through this without experimentation and actually starting to write some spec.
WASI might be implicated because it might be requested to provide a file descriptor used as the RPC transport buffer/socker/file. In the prototype I'm doing, the RPC read/write operations will need such file descriptor. But a more standard out of the box might not.
Assuming the interface between the user land code and the RPC code is a file descriptor is premature.
@aardappel any other thoughts why WASI might be the more appropriate layer for this?
Some notes about the potential interactions between WASM RPC and WASI: the RPC transport layer could be made available as a file descriptor to an RPC specific device file such as /dev/wasi/rpc
. Then:
read()
operation (which are already part of WASI as far as I know) would then be used to read then some WASM RPC code would deserialize the function calls as WASM/WAT + Interface Types for its arguments.write()
.The above would work, assuming that the WASI standard/implementation:
/dev/wasi/rpc
(or any other path) RPC interface device.My understanding is that 1. would fit in the scope of WASI. But what about 2. ?
Additional notes about the source code RPC "bindings".
I've prototyped the following in Rust:
// a function with an empty body will be wrapped as an outbound RPC call
#[wasm_rpc]
fn sub(a: i32, b: i32) -> i32 { }
// a function with an empty body will be wrapped as an inbound RPC call
#[wasm_rpc]
fn add(a: i32, b: i32) -> i32 {
a + b
}
that will generate the following WASM:
(func $import0 (import "env" "__wasm_rpc_sub") (param i32 i32) (result i32))
...
(export "__wasm_rpc_add" (func $func3))
...
(func $func3 (param $var0 i32) (param $var1 i32) (result i32)
The generated wrappers are - semantically - similar to the Interface Types' adapter functions. Maybe:
@interface
annotations should be used as RPC interfaces.@rpc-interface
annotation that builds on top of/derives the @interface
custom annotation.For the outbound RPC call/wrapper, we need to figure out:
40X
and 50X
status codes)async
: how would that translate into WASM? "async" is not mentioned in the WASM spec. Implying that the async nature of the RPC call will have to be - by default - hidden by the WASM VM layer.This seems like a really cool project, but it's not a great fit for the WebAssembly core spec. There are a few WebAssembly-related specifications. The core spec just describes the syntax and semantics of WebAssembly modules themselves, without specifying any other behavior for embedders. The JS API and WASI specs are layered on top of the core spec and provide more details about embedder functionality. A WebAssembly RPC spec would probably also be a separate spec layered on top of the core spec or perhaps the WASI spec.
The above would work, assuming that the WASI standard/implementation:
- Requires to provide the intended
/dev/wasi/rpc
(or any other path) RPC interface device.- Requires to expose some (specified? automagically listed?) WASM function symbols and bridge this RPC device file descriptor with an HTTP2 server.
My understanding is that 1. would fit in the scope of WASI. But what about 2. ?
WASI does not say anything about HTTP servers in the embedder, so you would probably want this to be a separate specification. The best way forward for this project is probably in its own github repo or organization, since it's layered on top of the technology being developed here in the WebAssembly github organization.
Hello,
First of all this is the first time that I write this kind of issue/proposal. So please excuse my bluntness or anything that might be missing/lacking. I'll gladly complete this post through comments and edits.
Considering:
one might think we have a portable/standard way to define a Remote Procedure Call (RPC) system.
By comparison to a well-known industry leader such as gRPC:
With the advent of FaaS and WASM based sandboxed FaaS (ex: CloudFlare Workers) and FOSS FaaS solutions in general (ex: OpenFaaS), a "native" RPC support in WASM would unlock scalable distributed computing without the limitations of the actual source programming language: C++ code bases could distribute workloads to Rust or Python code as long as the compiler/interpreted supports this WASM RPC standard. And that workload distribution would be based on:
Another way to see it is share-nothing dynamic linking with a HTTP2 transport in the middle. Providing the benefits of modular design, while adding also the possibility for distributing the workload.
I have prototyped it by:
#[no_mangle]
+extern "C"
and a bit of custom-mangling magic (a__wasm_rpc_
prefix really) to expose the RPC functions from a Rust code base.wasmtime
module loading to find those function symbols.This PoC will desmontrate that it is possibly doable without native support in the WASM standard. But without the compiler/standard support requirement, it means:
IMHO point 2 is a deal breaker. Thus my question about a potential WASM RPC standard proposal.
I would like to know if the above:
Please let me know if you have any question.