Add WASM/Wazero support

[benallfree]

@gedw99 suggested swapping out goja for wazero. The benefits cited were:

• faster startup and execution
• ability to write serverless functions in any language

There are many WASM compilers various languages:

• Rust
• Go
• C
• Python
• AssemblyScript (Typescriipt-ish)

However, Javascript itself - as well as any other dynamic language - cannot compile directly to WASM. Instead, it must be run by a WASM-based Javascript interpreter:

• Duktape - an embeddable Javascript engine, with a focus on portability and compact footprint that's capable of being run in the browser via WebAssembly.
• Javy - a JavaScript to WebAssembly toolchain, capable of generating WASI-compatible modules from JS by embedding the QuickJS engine.
• SpiderMonkey - experimental Rust bindings and generic builtins for SpiderMonkey for building WASI-compatible modules from JavaScript.

Or, we could continue using goja as the JS interpreter of choice. This seems like it might be the best route, unless it can be shown that a WASM-based JS VM is more performant than goja.

In any case, the biggest hurdle to WASM support is that we would need to create explicit bindings, effectively introducing a "serverless API".

Currently using goja, I can pass object references and pointers back and forth between Go and JS. goja makes that easy and pretty seamless. But with WASM, the interface feels more like an old style C function interface or a REST and webhook API because all you can really do is pass numeric and string values back and forth.

Example: It would not be possible, for example, to pass an Echo route context object into WASM and simply register an anonymous function handler that calls response.Send(200, 'Hello world'). Instead, the WASM environment would need to register a hook callback with OnBeforeServe in the host environment and, when that hook is called, register ANOTHER host callback for a route, which in turn calls ANOTHER hook callback when the route fires:

// TS serverless function example

// A lookup table of callbacks into the WASM environment
let callbackId = 0
const callbackLookup = {}
const addCallback = (name:string, context:any, cb: any) =>{
   callbackId++
   callbacks[callbackId] = { name, context, cb }
   return callbackId
}

// Register a given callback with the host environment
const registerInteropCallback = (callbackId: number) =>{
   const callback = callbackLookup[callbackId]
   const interopPayload = JSON.stringify(callback)

   // This calls into the host environment with a known payload format
   // The payload format includes an event name and context data that
   // varies based on the type of event
   // It is responsible for hooking into the specified event in the host
   // and behaving according to the context data passed
   _interop_addCallback(interopPayload) 
}

// This is exported from WASM. The PocketBase Go host environment knows that every WASM serverless plugin
// exports this function and expects a specific type of JSON payload to accompany each callback depending
// on what type of callback was registered
export const _interop_executeCallback = (callbackId: number, jsonPayload: string)=> {
   // Execute the WASM callback and return true for success, false for failure
   const res = callbackLookup[callbackId](JSON.parse(jsonPayload))
   return res
}

// Here is the user-level code
addCallback('OnBeforeServe', (e:OnBeforeServeEventData)=>{
   const myCustomMiddleware = createCallback(`EchoMiddleware`, middlewareInteropContext =>{
      return false
   })
   addCallback(`OnRoute`, {
      method: 'GET',
      path: '/api/foo',
      middlewares: [ myCustomMiddleware ]
   }, routeInteropContext => {
      console.log(`Do something here to create a host-level Echo response. The rabbit hole just goes deeper`)
   })
})

I'll leave this issue open for discussion. My opinion is that supporting WASM is a cool idea without a clear use case. Is anyone begging to write PocketBase hooks in Rust?

Tagging @ganigeorgiev for thoughts too.

[ganigeorgiev]

I haven't got the time yet to research the topic of Go interpreters vs WASM/gRPC plugins and other options in more details, so I don't have any specific opinion on this, but I agree that the WASM integration may end up too verbose and hard to use.

I don't see anything wrong with using goja (I also plan to use it for the triggers/workflow builder in the future); performance and resource-wise it is hard to tell which of the options will be better but I wouldn't base my choice only on this. For this type of package the DX is more important and If the developers really need "optimal" performance they can always just extend PocketBase using the Go event hooks.

[benallfree]

Good points @ganigeorgiev.

Also tagging goja author @dop251 for thoughts. He has lots of Go/JS interop experience.

[gedw99]

I don’t want to swap out goja. Js is awesone. I was suggesting adding Wazero engine. So you can use js or wasm.

[gedw99]

Wasi to be precise. You annotate exported functions just like with js

[gedw99]

You might find this combo of Javy , golang and WASM and Wazero useful.

It shows how js can be compiled to wasm and run with Wazero

golang code: https://github.com/Boostport/mjml-go

mjml code: https://github.com/mjmlio/mjml

docs: https://golangscript.com/g/compile-mjml-to-html-directly-in-your-go-applications

[benallfree]

@gedw99 This is still new to me, but as I understand Javy, it runs JS using an underlying QuickJS interpreter written in WASM. So a JS interpreter in WASM, running JS inside it.

capable of generating WASI-compatible modules from JS by embedding the QuickJS engine

Sorry for my imprecise wording about swapping out goja. I understand your original email better now too. You are suggesting adding WASM support to open the serverless options to other languages.

It still seems difficult to achieve interop between Go and WASM at the level PocketBase would need, but I do think the WASM idea could be beneficial if we get enough feedback that people want to write serverless functions in languages other than JS/TS.

[gedw99]

There is a higher level Wazero project called Capsule . It shows how the interop can be .

there is also Inngest, that uses docker and Wazero ( but turned off ).

These are all designed to react to events and so can be suitable to react to the CRUD events that Pocketbase exports.

Maybe they help if only as a reference. If you need links just holler :)

[gedw99]

If I get something working will holler :)