Does mlua want derive macros?

[Mikadore]

TL;DR

I want to add proc macros for deriving FromLua & ToLua, and something like lua_function (akin to pyfunction) to improve writing Lua modules. Maybe more, TBD. I'm prepared to write the code myself (in accordance to this crate's standards, of course)

Background

I'm writing a pure Rust Lua 5.4 interpreter. For that, I had imagined I'd also write a rust crate wrapping Lua (and optionally plugging in my interpreter as the "backend"), but I first looked at existing Lua bindings for rust. I really like mlua & it's design, so I decided to stick to it for any potential work of that kind, and hopefully upstream useful changes. As stated in the tldr section, I am prepared to implement all proposed functionality, if my proposal is desirable by the maintainers.

Proposal

At minimum, a FromLua and ToLua derive proc macro should be added, to be able to derive FromLua and ToLua using #[derive(ToLua, FromLua)] for structs where all fields implement FromLua and ToLua respectively. This would be a huge quality of life improvement, especially when writing modules for lua. The exact semantics of these macros is up to debate (e.g. unit structs or enums pose nontrivial considerations), my vision was just generating code that will take/return a Lua table with the field names corresponding to Rust's, and in the case of tuple structs something like _0, _1, ..., _n. Further potential features could be supporting methods by generating metatables from impl blocks (although there are some technical issues here with multiple impl blocks), and helper macros to rename the field in Lua.

The second part of my proposal is extending lua_module to remove most boilerplate when writing Lua modules in Rust. I think the lua_module macro itself is Ok returning a Result<Table>, but writing functions for use in lua should be simpler. I propose adding a lua_function (example name) attribute proc macro, that allows writing normal rust functions, where the arguments are all FromLua and the return type is ToLua. Optimally, if the function's first argument is of type &Lua, this macro should recognize that and pass it the invocation's instance. The way to use this function then would be through a macro (not a proc macro in this case), e.g. wrap_fn that translates the equivalent of lua.create_function(generated_fn) (but since this would be part of the crate we could just generate code for directly creating a Function instance, and then wrap_fn would use that).

This issue proposes a superset of #163, which could be closed if this proposal is accepted.

Example

I have implemented a very basic mockup of ToLua and FromLua here. Usage boils down to:

#[derive(Debug, FromLua, ToLua)]
struct Cookie {}

#[derive(Debug, FromLua, ToLua)]
struct Megastruct { ... }

#[derive(FromLua)]
struct Config {
    data: Vec<u8>,
}

fn get_data(_lua: &Lua, args: (Config,)) -> Result<Megastruct> {
    Ok(Megastruct { ... })
}

fn print_data(_lua: &Lua, args: (Megastruct,)) -> Result<()> {
    println!("{:#?}", args.0);
    Ok(())
}

#[lua_module]
fn moonbind(lua: &Lua) -> mlua::Result<mlua::Table> {
    let exports = lua.create_table()?;
    exports.set("get_data", lua.create_function(get_data)?)?;
    exports.set("print_data", lua.create_function(print_data)?)?;
    Ok(exports)
}

The accompanying Lua script:

moonbind = require('moonbind')

print('Getting data: ')
data = moonbind.get_data({
    data = {1,2,3,4}
})
print(data)
for k, v in pairs(data) do
    print(string.format("Key '%s' has type '%s'", k, type(v)))
end
-- Steal cookie >:)
data.maybe = nil
print('Got data: ')
moonbind.print_data(data)

Example output of running cargo build -q && cp ./target/debug/libmoonbind.so moonbind.so && lua main.lua:

Getting data: 
table: 0x565463e79400
...
Got data: 
Megastruct {
    ...
}

(I replaced all the fields/types in the source code/output for brevity, the readme in my repo has the complete output and code.

Motivation

I love proc macros. I especially love derive macros. They make everything simpler and cleaner. I think this would be of immense benefit to mlua, and since mlua_derive/macros is already a feature flag, it would come with no cost for users who aren't interested in them. For an example of more or less the exact same thing in practice, attribute macros are core for writing python modules using pyo3.

[khvzak]

Thanks for the detailed proposal!

I don't have any objections to implement FromLua / ToLua derive macros. Moreover it's included to my plans for the stable 1.0 release!

I propose adding a lua_function (example name) attribute proc macro, that allows writing normal rust functions, where the arguments are all FromLua and the return type is ToLua.

I quite like the approach implemented by rhai (See Engine::register_fn). Ideally would be preferable to register function "natively" without proc macros (if possible of course!) I was playing around that area a while ago but it never left the experimental stage.

-- I also like the pyo3 and have a draft (locally) to simplify registeting user data types using proc macros inspired by pyo3.

[lenscas]

For what its worth, I made a macro to implement ToLua and FromLua some time ago (it also does some other stuff related to my own crate but that should be easy to remove when porting).

It currently works with structs which become tables and most enums.

For structs you can annotate individual fields to be converted to another type before passing it to lua, this happens through the From trait and it will also auotmatically convert back through it when converting from lua.

For enums how it converts depends on the kind of enum. Right now, only enums without inner fields and enums that contain tuples are supported.

Enums without inner fields get translated to simple strings. Enums with tuple fields get translated to UserData and contain quite a few methods to work with them and also creates some way for lua code to create new instances of the enum.

Feel free to take whatever from the macro to help.

Docs: https://github.com/lenscas/tealr/blob/master/src/mlu/to_from_macro_doc.md Macro: https://github.com/lenscas/tealr/blob/master/tealr_derive/src/from_to_lua.rs

[Mikadore]

Thanks a lot @lenscas!

@khvzak I've been a bit busy, but I'm starting to work on a first PoC implementation. I'll open a PR once it's in a state to where the basics work and we can discuss more nuanced implementation details/features

[lenscas]

Actually, I totally forgot I also have https://github.com/lenscas/tealr/blob/master/src/mlu/picker_macro.rs#L38

Which is a macro_rules! macro that allows you to easily create a type that is an enum in Rust and exposed as an union in Lua. It does however also come with its own trait pub trait FromLuaExact<'lua>: Sized { which is basically FromLua but when implemented promises that it won't try and convert lua values to other lua values to make the conversion to Rust and instead prefers to fail (So, don't try to convert a Number to a string if the Rust side expected a String, just make the FromLua conversion fail instead.)

Feel free to also take this to mlua if deemed useful

[tStreichenberger]

Just noticed this thread and wanted to check in on where things are at with this. For a separate project, I have implemented derive macros for ToLua and FromLua as well. As part of this work, I’ve developed a helper macro which introduces functionality for default fields in a manner similar to #[serde(default)]. I could also look into adding something like #[serde(rename)], if you think it’d be useful. I can’t share the source code unfortunately as its in a private repo but would be happy to collaborate on this if you’re still working on it