libffi support?

[TerrorJack]

We looked a bit into how to port libffi to wasm32-wasi, and it seems impossible using the capabilities given in the current WASI snapshot.

libffi provides two functionalities:

• the basic API: call a function pointer with a runtime-constructed argument list, without precise knowledge of that function's signature at compile time
• the closure API: generate a function pointer which is a closure that captures a given argument list

In pure WebAssembly without host assistance, the basic API can be simulated using pure C: pattern match on the argument type list, foreach case, coerce the function pointer from generic type to the specific type, then perform an indirect call. The number of cases and resulting object size grows exponentially with supported argument listlength, but a small length should cover most realistic use cases.

But there's no way to simulate the closure API. The host needs to do some code generation, then populate an empty table slot and return its index.

Is it possible to enhance WASI and add capability to support libffi? It's a common library that has a ton of use cases in the C landscape, not all of which can be patched away to shave off this dependency. It's also possible that this involves a WebAssembly proposal rather than a WASI one, feel free to close this issue in that case :)

[devsnek]

I don't remember exactly where but there has been discussion about adding instructions to wasm for compile and instantiate and whatnot, it sounds like that might be helpful here?

[sbc100]

This certainly sounds more like a core wasm proposal (or rather a missing feature in the current core spec), rather than anything to do with WASI. To illustrate this, such a feature would also be very useful in emscripten.

[TerrorJack]

This certainly sounds more like a core wasm proposal (or rather a missing feature in the current core spec), rather than anything to do with WASI.

Fair enough, I'll close this one for the time being.

To illustrate this, such a feature would also be very useful in emscripten.

IIUIC emscripten generated code relies on JS, and with JS comes unlimited power, and libffi has indeed been ported to emscripten before.

Anyway, I've come up with a trick to implement closure API in pure wasm32, after this is done in our work we'll write up something to share with community.

[sbc100]

As far as I know libffi has not been ported to emscripten before (most likely for the reasons stated in this issue). I suppose its conceivable that it could be done in JS but I'm not away that anyone has done such a think. Even if it was possible using JS, a solution based in pure WebAssembly would always be preferable.

[williamstein]

TerrorJack:

libffi has indeed been ported to emscripten before.

sbc100:

As far as I know libffi has not been ported to emscripten before

In case anybody else stumbles on this thread and is confused by the above, there seems to be a port of libffi to emscripten here:

https://github.com/hoodmane/libffi-emscripten

It's sufficiently complete that Pyodide was able to use it to build the ctypes module in cpython, and I think they have all the tests passing. If you type the pyodide shell here and type import ctypes it is using libffi in emscripten:

https://pyodide.org/en/latest/console.html