Standard Library: FFI

[Sainan]

Something like Emscripten's API is probably not a bad idea, except without any marshalling for functions because I don't have a good (platform agnostic) way of doing that.

local ffi = require "pluto:ffi"
local user32 = ffi.open("user32")

local MessageBoxA = user32:wrap("MessageBoxA", "int", { "int", "string", "string", "int" })
MessageBoxA(0, "Hello from Pluto", nil, 0)

user32:call(
    "MessageBoxA", -- function name
    "int", -- return type
    { "int", "string", "string", "int" }, -- argument types
    { 0, "Hello from Pluto", nil, 0 } -- arguments
)

[cattokomo]

I'd recommend using TCC API or libffi.

[Sainan]

My concern is compat with WASM, where W^X is baked into the ISA, by which I mean new code to run cannot be allocated at runtime without a bunch of coordination.

Otherwise, I could definitely allocate an executable region of memory and spit some x86 or ARM bytecode into it, but that approach is obviously extremely non-portable — no matter how many fancy libraries and wrappers you use for it.

[cattokomo]

You could probably just left out FFI as missing features in the WASM builds. But it's not a great option either. If we can just implement an FFI thar relies on WebAssembly JS API without needing Emscripten API.

[Sainan]

I think just getting an API working is the first priority. I have tons of ideas for possible improvements beyond the initial API proposed here, but it will have to be guided by the needs of actual usage.

[cattokomo]

By "getting an API working", did you mean getting it working on all platforms including WASM?

[Sainan]

Of course. Our standard is to have a consistent standard library across platforms, otherwise there's diminishing value in adding it to the standard library to begin with. I think the only time we straight-up gutted a library from a platform is 'socket' which is simply not feasable on WASM due to web sandboxing, and there's still plenty of value in having a socket library that works on Windows, Linux, & co without needing a DLL/SO for it.

As for FFI, I'm not sure what the use cases for it would even look like, which is why I am hesitant on trying to solve for specific problems. The problem here is, of course, usage of the 'ffi' library would "never" be truly portable because you might be able to ship a .dll and have it run on most Windows machines, but good luck trying to ship a .so that works on anything close to a majority of Linux machines. And for libraries that are expected to already be present on the OS... yeah, that's already killing your chances of running this code on another OS.

All of this to say: I don't want to make any assumptions and simply offer an 'ffi' primitive to at least alleviate some friction in regards to doing FFI in Lua/Pluto.

[Apis035]

When/what version of Pluto is expected to come with FFI implemented?

[Sainan]

None right now.

[Sainan]

An additional consideration would be structs, similar to what PHP does:

<?php
// create gettimeofday() binding
$ffi = FFI::cdef("
    typedef unsigned int time_t;
    typedef unsigned int suseconds_t;

    struct timeval {
        time_t      tv_sec;
        suseconds_t tv_usec;
    };

    struct timezone {
        int tz_minuteswest;
        int tz_dsttime;
    };

    int gettimeofday(struct timeval *tv, struct timezone *tz);    
", "libc.so.6");
// create C data structures
$tv = $ffi->new("struct timeval");
$tz = $ffi->new("struct timezone");
// call C's gettimeofday()
var_dump($ffi->gettimeofday(FFI::addr($tv), FFI::addr($tz)));
// access field of C data structure
var_dump($tv->tv_sec);
// print the whole C data structure
var_dump($tz);
?>

Tho I'm not sure if we want to steal their API.

[cattokomo]

Isn't that essentially LuaJIT FFI in PHP?

[Sainan]

Possible combined API design:

local ffi = require "ffi"

-- Using a shared library (DLL)
local user32 = ffi.open("user32")
user32:cdef[[
int MessageBoxA(void *w, const char *txt, const char *cap, int type);
]]
user32:MessageBoxA(0, "Hello, world!", "My Pluto Script", 0)

-- Using structs
ffi.cdef[[
struct colour { int r; int g; int b; };
]]
print(ffi.sizeof("colour")) --> 12
print(ffi.offsetof("colour", "r")) --> 0
local col = ffi.new("colour")
col.r = 255
print(col.r) --> 255