Convenience functions taking advantage of zig's comptime

[VisenDev]

I think it would be nice if this library could provide some functionality that would automate some of the tedious work writing lua bindings.

For example, theoretically, it should be possible to provide an api something like this

fn autoCall(lua: *Lua, comptime ReturnType: type, function_name: []const u8, args: anytype) !ReturnType;

const result = try lua.call(i32, "add", .{1, 2});

It should be possible to iterate over the args at comptime to make sure only types that can be represented in lua are passed. Then an inline for loop can be used to iterate over the args and call the appropriate lua_push function.

Another example would be get and set functions for lua globals

fn get(lua: *Lua, comptime T: type, name: []const u8) !T;
fn set(lua: *Lua, name: []const u8, value: anytype) !void;

It might also be possible to automatically write the bindings for a zig function to be called by lua at comptime?

I'm not entirely sure how this would work. Here's some sort of pseudo code of what I was thinking about. But some more thought about how to make this work within the current limits of zigs comptime would still be needed. You would probably need to then generate a second interface function that has the proper pop functions present so that it can properly call the zig function.

Not really sure but these are just some ideas

fn autoSetFunction(lua: *Lua, name: []const u8, comptime function: anytype) !void {
     const type_info = @typeInfo(@TypeOf(function));
     if(type_info != .Fn) @compileError("you must pass a function pointer");

     //create an interface function based off of the parameter types
     inline for(type_info.function.params) |parameter| {
         switch(parameter.type) {
             i32, u32 => //etc
     }

     //then pass the interface function to lua
     lua.pushFunction(...);
}

[VisenDev]

Expanding on this, I think functionality which could provide an automatic way of parsing a lua table into a zig struct and/or a zig standard hashtable would also be really nice.

As for parsing into a zig struct, a really nice way to do this would be to convert the lua table to json, and then parse that json into a zig struct using the std.json library

I'd like to work on some of this myself, but I won't if there is no interest or guidance from the core dev team @natecraddock

[VisenDev]

Example implementation of automating the call of a lua function from zig as a proof of concept

pub fn autoCall(l: *Lua, comptime ReturnType: type, func_name: [:0]const u8, args: anytype) !ReturnType {
    if (try l.getGlobal(func_name) != ziglua.LuaType.function) return error.invalid_function_name;

    inline for (args) |arg| {
        switch (@typeInfo(@TypeOf(arg))) {
            .Int, .ComptimeInt => {
                const casted: ziglua.Integer = @intCast(arg);
                l.pushInteger(casted);
            },
            .Float, .ComptimeFloat => {
                const casted: ziglua.Number = @floatCast(arg);
                l.pushNumber(casted);
            },
            .Pointer => |info| {
                switch (info.size) {
                    .Slice => {
                        if (info.child != u8) {
                            @compileError("Only u8 arrays (strings) may be pushed");
                        }
                        if (info.sentinel != 0) {
                            @compileError("Strings must be sentinel terminated");
                        }
                        l.pushString(arg);
                    },
                    .One, .C => {
                        l.pushLightUserdata(@constCast(@ptrCast(arg)));
                    },
                    else => {
                        @compileError("only slices or single item pointers may be used as args");
                    },
                }
            },
            else => @compileError("Only ints, floats, strings, and pointers may be used as args"),
        }
    }

    const num_results = if (ReturnType == void) 0 else 1;
    l.protectedCall(args.len, num_results, 0) catch unreachable;

    switch (@typeInfo(ReturnType)) {
        .Int => {
            const raw = try l.toInteger(-1);
            const casted: ReturnType = @intCast(raw);
            return casted;
        },
        .Float => {
            const raw = try l.toNumber(-1);
            const casted: ReturnType = @floatCast(raw);
            return casted;
        },
        .Pointer => |info| {
            switch (info.size) {
                .Slice => {
                    if (ReturnType != []u8) {
                        @compileError("Only slices of u8 (strings) may be returned");
                    }

                    return try l.toString(-1);
                },
                else => {
                    return try l.toUserdata(-1);
                },
            }
        },
        else => {
            @compileError("invalid return type");
        },
    }
}

test "autocall" {
    const a = std.testing.allocator;
    var l = try Lua.init(a);
    defer l.deinit();
    //l.openLibs();

    const program =
        \\function add(a, b) 
        \\   return a + b
        \\end
    ;

    try l.doString(program);
    const sum = try autoCall(&l, usize, "add", .{ 1, 2 });
    try std.testing.expect(3 == sum);
}

[VisenDev]

After thinking about it some more, I think I've come up with a way to also automatically generate the interface for zig function that lua can call

Here's a barebones sketch of how this would work

fn GenerateInterface(comptime function: anytype) type {
    const info = @typeInfo(@TypeOf(function));
    if (info != .Fn) @compileError("function pointer must be passed");
    return struct {
        pub fn interface(l: *Lua) i32 {

            //somehow create an anon tuple here of all the values we get from lua
            var parameters: std.meta.ArgsTuple(@TypeOf(function)) = undefined;

            inline for (info.Fn.params, 0..) |param, i| {
                switch (@typeInfo(param.type.?)) {
                    .Int => {
                        const result = l.toInteger(-1) catch unreachable; //todo: something else with this error,
                        parameters[i] = @intCast(result);
                    },
                    else => {},
                    //etc...
                }
            }

            const result = @call(.auto, function, parameters);

            switch (@typeInfo(@TypeOf(result))) {
                .Int => {
                    l.pushInteger(@intCast(result));
                    return 1;
                },
                else => {
                    return 0;
                },
            }
        }
    };
}

pub fn foo(a: i32, b: i32) i32 {
    return a + b;
}

pub fn autoSetFunction(l: *Lua, function: anytype, func_name: [:0]const u8) void {
    const Interface = GenerateInterface(function);
    l.pushFunction(ziglua.wrap(Interface.interface));
    l.setGlobal(func_name);
}

test "gen wrapper" {
    const a = std.testing.allocator;
    var l = try Lua.init(a);
    defer l.deinit();

    autoSetFunction(&l, foo, "foo");
    try l.doString("foo(1, 2)");
}

[VisenDev]

@natecraddock Well I haven't heard anything back so I'm just gonna start working on this on my own fork in the meantime

[natecraddock]

Hey @VisenDev so sorry for not replying sooner. I haven't had the chance to look at this in detail, but I will sometime this week.

I will say though, that I'm not sure how I feel about this yet. I'm not entirely against adding these convenience functions, but I also want to be be careful with what gets added to Ziglua.

I first want to focus on getting Luau and Luajit added (Luau being nearly done now), and then polishing up documentation. After that is when I wanted to consider adding additional conveniences. Or perhaps delegating those to a separate package.

But as I said, I haven't really taken the time to look at this in detail. So feel free to work on this for the time being, and we will see where things go from there! Thank you for the ideas and interest in contributing!

[VisenDev]

Awesome, just wanted to make sure I wasn't getting missed or anything because this idea really interests me.

To help you think it thru, I've pretty much implemented all of the suggested features for lua 5.4 in my fork of this repo

Feel free to critique and suggest changes. I still believe there is a lot of potential for making ffi much easier given zig's unique comptime abilities

Additionally, I added some generic pushing and popping functions for the lua stack, as this made the implementation of wrapper functions much easier. But those don't need to necessarily be part of the public api

I also added some experimental functionality for automatically converting between lua tables and zig structs and vice versa

[VisenDev]

Update: I've also been experimenting with the ability to automatically generate the interface for a zig function that has a single upvalue used as the function's context.

Example:

const Bar = struct {
    a: std.mem.Allocator,
    bizz: i32,
    bazz: bool,

    pub fn foo(self: *@This(), a: i32, b: i32) i32 {
        return (a + b) * self.bizz;
    }
};

test "autoPushFunctionWithContext" {
    var lua = try Lua.init(testing.allocator);
    defer lua.deinit();

    var ctx = Bar{ .a = testing.allocator, .bizz = 10, .bazz = false };
    lua.autoPushFunctionWithContext(Bar.foo, &ctx);
    lua.setGlobal("foo");
    try lua.doString("local c = foo(1, 2)"); //upvalue context passed to zig automatically
}

I'm not really sure if this a good idea or not, or how exactly the api should be structured.

The use case is that in zig we have a lot of structs with functions, those functions are usually called via dot syntax. So any lua functions calling those zig functions have to have access to that context somehow, I have not figured out yet what the best way to do this is. But one method is to give the function an upvalue, which is what i've done here. The upvalue is then just always used as the functions first parameter.

Context for this use case is me designing a game scripting api for lua from zig

[natecraddock]

I finally had some good time today to really read this deeply. I like a lot of the suggestions here!

And I looked through your fork. I see (https://github.com/VisenDev/ziglua/commit/fd925dc0e4a33e43b368505b0bf7204e177f17c4) you also caught the bug where I didn't have the run test step setup properly, sorry about that!

I'm still not sure if I think this makes sense in Ziglua, or in a separate package. I'm leaning toward Ziglua, but once things have settled more.

I just took some time today to write down a roadmap to 1.0.0. I think this may give you some context on my plans for the future: https://github.com/natecraddock/ziglua/issues/32

[VisenDev]

I finally had some good time today to really read this deeply. I like a lot of the suggestions here!

Awesome, I'm going to continue development of my fork to see how things mature.

I am using my fork for the development of the lua api in my game writton in zig so I will be able to test things in a real world codebase and see how things go.

So far, I have pretty much entirely eliminated the need to manually write any interface functions, and I can pretty much just use my normal zig functions with the lua api now without manually writing any interfaces. This also leads to faster iteration time, as I can now refactor my zig code without also needing to refactor all the interface functions

[Interrupt]

My gamedev framework does automatic binding function binding like this using Ziglua, you can check out how I did it here: https://github.com/Interrupt/delve-framework/blob/main/src/scripting/manager.zig

[VisenDev]

I think this can be considered completed given that most of the features I requested have now been implemented. Polish and future changes can go in another issue