Extend package.searchers for Lua in Redbean

[shakna-israel]

Lua uses the table found at package.searchers to see how to require libraries.

It should be possible to implement a simple function so that you can require from inside the zip filesystem without any tweaks/workarounds on behalf of the programmer.

This isn't something I'm really familiar with doing, but from the Lua side it works to do:

package.searchers[#package.searchers + 1] = function(libraryname)
    -- Returns function that creates the library, and a string where it was found
    return load(LoadAsset(string.format("/%s.lua", libraryname))), string.format("/%s.lua", libraryname)
end

mylib = require "mylib"

The function should probably be extended to search based on the package.path to be compliant with Lua expectations (using the seperators found in package.config).

And translated to C, rather than spun up in Lua.

There's probably also an ordering thing to think through. Adding it as the last package searcher means that Lua will prefer the host OS, only trying the zipfs as the last option. That might be the preference, or it might be preferable that it tries the zipfs as the first option.

[shakna-israel]

Okay, to save a little bit of work, this is what a proper package searcher would look like in Lua. Just needs to be translated to the C-API and installed somewhere inside LuaInit, after luaL_openlibs:

package.searchers[#package.searchers + 1] = function(libraryname)
    -- Get the path seperator
    local path_sep = package.config:sub(3, 3)
    -- Get the name substitution
    local name_rep = package.config:sub(5, 5)

    local ret = {}
    local pattern = string.format("([^%s]+)", path_sep)
    string.gsub(package.path, pattern, function(path)
        -- Skip if we've already found it
        if #ret < 0 then
            -- Rewrite the path to fit:
            local proper_path = string.gsub(path, name_rep, libraryname)

            -- Attempt to load the library:
            local f = load(LoadAsset(proper_path))
            if f ~= nil then
                -- Set the function to return
                ret[1] = f
                -- Set the path we found it at, to return
                ret[2] = proper_path
            end
        end
    end)

    -- Return function and where it was found:
    if #ret > 0 then
        return ret[1], string.format("zipfs://%s", ret[2])
    else
        return nil
    end
end

The opinionated things this does:

• For the debugging string that says where a library was found, it prepends zipfs:// to the file path. To make things clear.
• This installs the Zip library loader as the last loader. So Lua will still prefer the host OS's file system over the zip.
• Assumes that the package.config symbols are single-byte, which is the default on every system (inc. embedded ones) that I know of. So that we don't have to implement a newline split for no reason.

Once that's done, you can use require as you normally would, and it'll search the inside of the zipfile as well, simplifying distribution a ton.

[jart]

Have you tried using the zip: filename prefix? Cosmopolitan bakes that pretty deeply into the C library, all the way down to the system call level. So any Lua functions for I/O that use that prefix should be directed to read from the zip structure.

[jart]

Another trick to try, which was suggested by someone brilliant on the mailing list, is as follows:

package.path = 'zip:?.lua'

https://groups.google.com/g/cosmopolitan-libc/c/1Deo0YsSGFc

[shakna-israel]

Didn't realise that zip paths was so deeply integrated.

In that case, would it possible to get something like this integrated by default?

package.path = package.path = package.path .. package.config:sub(3, 3) .. "zip:" .. package.path:gsub(package.config:sub(3, 3), package.config:sub(3, 3) .. "zip:")

[jart]

Your wish is my command.

I've updated redbean so the default package.path is zip:.lua/?.lua;zip:.lua/?/init.lua which is documented here: https://justine.lol/redbean/index.html#dotlua

A new redbean 0.4.5 release has been pushed to the website, which contains this improvement: https://justine.lol/redbean/index.html You may also view the live demo at http://redbean.justine.lol