search

WebAssembly / module-linking

Proposal for allowing modules to define, import and export modules and instances
Other
120 stars 13 forks source link

Library on demand (LOD) in WASI #38

Open vasuskimo opened 2 years ago

vasuskimo commented 2 years ago

LOD, library-on-demand is a cross between a library and a service using Web Assembly WASI. LOD obviates the need for linking during compile time. Simply call LOD, like a web service when you need and it will be downloaded from some repo and when you are done simply release it.

There is a precedence in Javascript/node where one can do this kind of dynamic import.

do "import("https://foo.bar") or i="include("https://foo.bar")

This LOD feature would become an imperative as WASI begins to gain traction.

lukewagner commented 2 years ago

To a first approximation, it sounds like what you want is ESM-integration, but restricted to wasm modules (i.e., not allowing JS modules). Depending on what your use cases are, you might need ESM-integration + Module Linking. In Node.js, I don't believe you can simply import a random URL, but I think in Deno you can, and it seems fairly unrestricted (although I expect configurable). So once Deno implements ESM-integration, you'd be able to basically do what you're talking about, without any additional standards. One could also implement compatible behavior, minus the JS VM, using a wasm engine like Wasmtime, and so you could imagine creating a portable standard for this (much smaller) ESM-integration-minus-JS behavior.

A question I have, though, is whether there are a wider variety of hosts that would want to implement this same "fetch live modules via URLs" behavior. For browsers it makes a lot of sense, and maybe for Deno as well. But in other production server environments, I've seen the need for using a more restricted, locked-down way of naming modules that avoids live fetches from third-party URLs. So I'd personally be interested to learn about more concrete examples beyond browsers and Deno.

vasuskimo commented 2 years ago

Imagine you are building a bunch of microservices in any language, that link to open source github repos instead of linking libraries. You simply define the library you want in a "bill of materials" xml file without having to link before its use. Things would be lot cleaner. ...and then you could build a "walled garden" by using yet another open source (which might not exist today) and specify that, only include libraries that are "blessed".

rossberg commented 2 years ago

Perhaps tangential to the question, but my 2 cents re URLs: Independent of the use case, URLs (or rather URIs) are the standard for identifying resources. They are universal, fairly well-defined and extensible. Consequently, I think they are the right choice for identifying external modules, regardless of which schemas an application actually supports (using URLs doesn't imply that http: is allowed). URLs avoid reinventing the wheel and they are future-proof.

That said, it's annoying when applications go for URLs but then do not respect their semantics – especially omnipresent on planet JavaScript, which has a history of abusing relative URL syntax and assigning different meaning to the relative URLs "a" and "./a", in plain violation of the URL specification. :(

vasuskimo commented 2 years ago

The #log4j vulnerability is a classic use case where LOD could be used to react to bugs efficiently. Today, organizations across the world are working like headless chicken to not only identify the software which has this vulnerability but also upgrade with a later version of log4j or logback etc., We would totally avoid all that unnecessary effort, anxiety, stress and time spent by LOD. I rest my case.