Embedding Lua

[bpj]

@CDSoft is embedding Lua something you might consider? Not only because it would be always available but even more because there could be a persistent Lua environment.

[CDSoft]

I'm not considering embedding Lua. Do you have any use case in mind? I think a Pandoc filter (in Lua) would do the job (unless you need specific interactions with pp).

[bpj]

I would say that pp and Pandoc filters are good for different things. I probably could do everything I do with pp with filters but not as easily. In particular it is often preferable to work with strings rather than Pandoc elements, and then have Pandoc parse those strings — i.e. to use a preprocessor. In doing so I often need string manipulation like substitution (by regex usually). I currently use pp's !lua macro or perl via pp's !sh macro. It works of course, but having Lua embedded in pp would have several benefits (in ascending order of importance):

1. You could rely on Lua always being available when pp is available. Not so much an issue on *nix, but very good on Windows.

2. There could be a persistent Lua environment throughout the pp run, so that you could import or define functions, either as global functions or as members of a global table, and variables at the beginning of the pp run and then use them elsewhere. Now a new instance of Lua (or another interpreter) is spawned every time, and you have to define all data and functions in every !lua block, which is not only inconvenient and wasteful, but also makes it impossible to keep state (which is a problem Pandoc filters also have.)

3. Ideally it would be possible to define pp macros as Lua functions, with arguments to the pp macro being passed as proper (string) arguments to the Lua function. That would not only make for more readable and maintainable pp code, but would make sharing values between pp and Lua much more robust. Currently there is a problem doing the latter. In theory you can just say local foo = "!1" inside the Lua block, but what if the macro argument contains double quotes!? In practice you have to use a Lua long string — preferably with a lot of = characters in the delimiters. (This has the concomitant problem that you can't use escapes like \n in the string. However that is not a huge problem: in fact using a Lua table/gsub to resolve escapes allows me to define \s as a custom escape for the space character, but then I would like to have the escape resolving function — and the substitution table — persistent, which brings us back to 2!) Currently inside !lua blocks I use an unholy mess of pp macros defining Lua code and actual Lua code, i.e. what gives preprocessors a bad name.

[CDSoft]

Sorry for the late reply... Please keep in mind that pp is not supported anymore, it's hard to deploy. For new projects I suggest ypp which is is based on a Lua interpreter and way easier to compile and install and binaries are easier to produce (thanks to zig) and deploy (see hey).