[Discussion] Lockfile syntax

[vhyrro]

Basics

This issue serves as a discussion for a lockfile format.

The lockfile is supposed to be the most versatile source of truth in rocks. First, I'd like to enumerate all features that such a lockfile should support, then I'll lay out a basic proposal syntax.

At its core, a lockfile serves as a snapshot: it allows developers to all be on the same page by pinning dependencies to a given version (with the option to update them with rocks update). A nice side effect of lockfiles is the ability to make the environment reproducible, useful for tooling like Nix, which has had a notoriously bad time with luarocks prior because of suboptimal amounts of data stored in the lockfiles.

Thus, tooling should contain the following data:

• All rocks installed in the current tree (including dependencies of each rock)
  • The name of each rock
  • The version of each rock

Apart from this, we would also like to support pinning of rocks. When a rock is pinned, its dependencies should also be pinned. Therefore, every rock should also contain a pinned boolean field.

Cargo.lock contains the source the rock was pulled from (git or otherwise), but in Lua land the source is embedded in the rockspec of each rock, so we don't need that.

The Format

I propose that we use JSON to store our lockfile data. toml is fit for editable data, but lockfiles are not designed to be user-writeable. JSON is also absurdly fast for read/write operations.

Here's a sample file:

{
    "version": "0.1.0",
    "rocks": {
        "f0509dab05065162bacae5eaba5cafd93a06369911fada7b7d2a33c794b6b5ae": {
            "name": "neorg",
            "version": "0.1.0",
            "pinned": false,
            "dependencies": [
                "94be53125e66d7713f5545a92857666ff456f1bd7ca65edb57d0c0a43dfffe37"
            ]
        },
       "94be53125e66d7713f5545a92857666ff456f1bd7ca65edb57d0c0a43dfffe37": {
            "name": "lua-utils.nvim",
            "version": "0.4.0",
            "pinned": false,
            "dependencies": null
        }
    }
}

I can't think of anything else off the top of my head, but I do recall there being extra features that we were discussing. That's why I'd like to enumerate them all in a single issue.

[mrcjkb]

Here are some ideas (based on the output of luarocks-nix):

• URI of the rockspec (if present - we may want to support installing packages without rockspecs, e.g. for Nvim plugins).
• sha256sum of the rockspec (to account for the possibility of someone force uploading a rockspec to luarocks.org).
• If no rockspec is present, the source URI.

Since lockfiles aren't a thing in the luarocks ecosystem, we will also need to include transitive dependencies if a dependency doesn't have a lockfile. Otherwise, we can just merge a dependency's lockfile into the one we're creating.

[vhyrro]

About the sha256 sum, I was wondering if that should be the unique key for the given rock. Each rock is then referenced by this sha exclusively to speed up lookups, I think that's pretty interesting.

However, the sha key should then contain the name and version of the rock, not just the sha of its content. Not sure if that's a hindrance in any way.

About the URI - do you have an example of the source URI in question? Can't quite wrap my head around whether it should be a URI to https://luarocks.org/the/rock or a file:// uri.

[mrcjkb]

About the sha256 sum, I was wondering if that should be the unique key for the given rock. Each rock is then referenced by this sha exclusively to speed up lookups, I think that's pretty interesting.

It sounds like an interesting idea. I cant think of any issues off the top of my head and it might be useful if we want to support multiple versions of the same dependency.

About the URI - do you have an example of the source URI in question? Can't quite wrap my head around whether it should be a URI to https://luarocks.org/the/rock or a file:// uri.

It could be either, imo. Of course, if it's a file:// URI, you will only be able to build it locally, but that's fine.

[vhyrro]

Suggestion: the lockfile can easily be used as a source of truth of all currently installed plugins (without having to resort to slower filesystem reads). However, we can't just enumerate all keys in the JSON, since that'll include all transitive dependencies too.

This is desirable in some cases, but in other cases you only want to know of the main rocks that are installed (disregarding dependencies). But, trying to calculate which plugins are transitive dependencies can also prove to be complex.

For this reason, we might want to have an entrypoints in our lockfile:

{
    "version": "0.1.0",
    "entrypoints": [
        "f0509dab05065162bacae5eaba5cafd93a06369911fada7b7d2a33c794b6b5ae"
    ],
    "rocks": {
        "f0509dab05065162bacae5eaba5cafd93a06369911fada7b7d2a33c794b6b5ae": {
            "name": "neorg",
            "version": "0.1.0",
            "pinned": false,
            "dependencies": [
                "94be53125e66d7713f5545a92857666ff456f1bd7ca65edb57d0c0a43dfffe37"
            ]
        },
       "94be53125e66d7713f5545a92857666ff456f1bd7ca65edb57d0c0a43dfffe37": {
            "name": "lua-utils.nvim",
            "version": "0.4.0",
            "pinned": false,
            "dependencies": null
        }
    }
}

[mrcjkb]

linking #20