Feature request: Pinning TOML plugins

Phault commented 6 months ago

Currently it's possible to achieve this, if it's hosted on a forge like GitHub which you trust, then you can create a permalink to the current version of the plugin e.g:

https://raw.githubusercontent.com/Phault/proto-toml-plugins/51e5ec7f5bf0ea380a9665652ca7cb40f8dcb76c/act/plugin.toml

However this is tedious, not obvious, and difficult to document in your plugin's README, as it changes on each commit. So right now the common method is just to point it to the latest version which is a glaring security hole, as at any point the plugin author can make the tool's download-url point to a malicious payload.

I don't see a general way to solve this for the source locator, so I think discouraging its use and implementing a native git locator would be better.

Ideas

1) `github` locator should support non-releases and TOML plugins

Currently it assumes you're interested in releases and only looks for archives or wasm assets in that release. Not sure if the archive can contain a TOML plugin, but using releases for TOML plugins seems a bit overkill in any case.

I don't think this one makes sense, so quickly jumping further to...

2) Implement a `git` locator

I'm thinking the syntax could be:

git://[<user>[:<password>]@]<hostname>[:<port>][/]<path>[?head=<branch-or-tag>][&commit=<sha>][&path=<subpath>]

Whether it should start with git:example.com or git://example.com I'm not sure about, the latter does however read clearer if you use basic auth. It's inspired by how npm handles it. git+ssl://user@example.com should probably also be possible.

Examples:

# resolves to latest commit on the default branch
git://github.com/Phault/proto-toml-plugin

# resolves to latest commit on head
git://github.com/Phault/proto-toml-plugin?head=main
git://github.com/Phault/proto-toml-plugin?head=v1.0.0

# resolves to a specific commit
git://github.com/Phault/proto-toml-plugin?commit=06506040433a6457bc64b457dfabce01678f33be0354

# resolves to the given commit, but head will be used to query for updates
git://github.com/Phault/proto-toml-plugin?head=main&commit=06506040433a6457bc64b457dfabce01678f33be0354

# with subpath
git://github.com/Phault/proto-toml-plugin?path=act/plugin.toml
git://github.com/Phault/proto-toml-plugin?head=feature-branch&path=act/plugin.toml
git://github.com/Phault/proto-toml-plugin?commit=06506040433a6457bc64b457dfabce01678f33be0354&path=act/plugin.toml

This in itself wouldn't solve the issue, but it could if the proto plugin add was made the preferred way to add plugins, and it additionally used the locator to transform into an explicit URI similar to proto pin's --resolve option like so:

proto add plugin act "git://github.com/Phault/proto-toml-plugin?path=act/plugin.toml"
proto add plugin act-beta "git://github.com/Phault/proto-toml-plugin&head=beta?path=act/plugin.toml"

Resulting in the .prototools entry:

[plugins]
act = "git://github.com/Phault/proto-toml-plugin?commit=ac06506040433a6457bc64b457dfabce01678f33be0354&path=act/plugin.toml"
act-beta = "git://github.com/Phault/proto-toml-plugin?head=beta&commit=553ae7da92f5505a92bbb8c9d47be76ab9f65bc2&path=act/plugin.toml"

milesj commented 6 months ago

The primary reason I haven't added git support, is that cloning the repo to extract the file adds considerable overhead.

At the top of your post, you're talking about how source: URLs have some specific issues, but your examples kind of have all the same issues. IMO they're just as verbose and confusing, but less explicit than an absolute URL, as it's trying to mimic what the URL already does (the path, the commit sha, etc).

Phault commented 6 months ago

The primary reason I haven't added git support, is that cloning the repo to extract the file adds considerable overhead.

Yeah that makes sense. Perhaps doing some trickery similar to https://www.npmjs.com/package/hosted-git-info might get you most of the way.

At the top of your post, you're talking about how source: URLs have some specific issues, but your examples kind of have all the same issues. IMO they're just as verbose and confusing, but less explicit than an absolute URL, as it's trying to mimic what the URL already does (the path, the commit sha, etc).

Unless some knowledge about e.g. the githubusercontent.com domain is built into the source locator, then I don't see how it would be able to transform a unpinned https://raw.githubusercontent.com/Phault/proto-toml-plugins/main/act/plugin.toml url into a permalinked one such as https://raw.githubusercontent.com/Phault/proto-toml-plugins/51e5ec7f5bf0ea380a9665652ca7cb40f8dcb76c/act/plugin.toml.

Even if it did, then it would lose the context of where it should look for plugin updates (if that became a thing later), plus needing special github logic for how to check for updates.

The part that actually solves my main issues with using source is the manual need for finding the permalink to use, hence the need some extra smarts in the proto plugin add command to automatically lock it.

milesj commented 6 months ago

Maybe we just expand github: to support more than releases: github:org/repo/path/to/config.toml, github:org/repo/path/to/config.toml#branch, etc.

Phault commented 6 months ago

Yeah I think you're right, that would get us far in a short time. Support for other git hosts could come later.

Also no need for relying on "proto plugin add" to automatically add the commit shas I mentioned, as that could go in the lockfile we spoke about.

moonrepo / proto