Improve published release artifact names

sagikazarmark commented 5 months ago

It's not very easy to build automation with the current naming scheme of release assets.

One issue is the version in the archive name: GitHub has this useful feature where you can go to a latest release by a special URL: https://github.com/caddyserver/xcaddy/releases/latest

Unfortunately, there is no way to download artifacts without knowing the version (since it's part of the name).

By removing the name, one can download artifacts for the latest version without actually knowing what it is.

Another mild annoyance is the OS name for macOS builds: all of the other targets use the "standard" Go OS (which would be darwin in this case). (This is easy to overcome; I just thought I'd mention it).

mholt commented 5 months ago

Hmm, I think of knowing the version as a feature actually. It's nice to know once you've downloaded an artifact what version you've got.

As for Mac/Darwin, I am always torn on it, because nobody knows what Darwin is. Heck, even I don't really know to this day. I'm sure I could look up some history. Similarly I get confused with i686 arches, etc (I'm glad Go doesn't use i686 as an arch identifier).

francislavoie commented 5 months ago

Changing this at this point would be a breaking change for the release processes of many platforms. We're not in contact with all of them and we can't really trust that we'll be able to communicate a change like this to them.

I think using "latest" is almost always a foot-gun. You should pin to a specific version number, and you should regularly update it to the latest version number as necessary. You can sign up for release notifications on GitHub (click watch, custom, check releases only) and update it in response to those notifications.

sagikazarmark commented 5 months ago

I think using "latest" is almost always a foot-gun.

Agreed, but hard-coding a fallback version is almost as big a footgun.

Using the GitHub API to figure out what latest resolves to is not always an option.

Hmm, I think of knowing the version as a feature actually. It's nice to know once you've downloaded an artifact what version you've got.

That's fair...when you download an archive manually. I don't know how often people actually do that. They either use a package manager or some other automation/script in my experience.

Incidentally, if there was a container image published for xcaddy, there would be a latest tag.

Changing this at this point would be a breaking change for the release processes of many platforms.

That's another fair point, although 1.0.0 hasn't been tagged yet which is what led me to open this issue: if there was a time to make a change like this, it's now.

I understand if it's a no though.

mholt commented 5 months ago

I just realized we're in the xcaddy repo :man_facepalming:

Anyway, I have less strong opinions about how we distribute xcaddy -- I can see how more of these downloads might be automated.

mohammed90 commented 5 months ago

We can probably add another build in goreleaser with another ID, e.g. latest, to produce artifacts latest in place of version. If we converge on yes, it can be added, but we'll have to validate how it impacts the release pipeline.

polarathene commented 1 month ago

Hmm, I think of knowing the version as a feature actually. It's nice to know once you've downloaded an artifact what version you've got.

What users source a Caddy build from GH releases in a way where the version in the asset name matters?

At the time of retrieval it's either latest or the release tag, that information is available.

If they're automating it, they could store the associated information to the filename themselves if it was something they considered important.
Others may just be wanting to grab the asset from the releases and dispose the original release asset they extracted (I just pipe via CLI to tar -xz myself), the version can then be queried from the binary with xcaddy version.

As stated, it prevents a "feature" of being able to use latest, which isn't something I'd use in a pipeline/workflow but is sometimes preferrable than the other suggested methods where go install is more overhead and I don't have to think about package manager and it having the version I'm after.

Simple as:

# URL:
# - Latest release: https://github.com/caddyserver/xcaddy/releases/latest/download/xcaddy_linux_amd64.tar.gz
# - By release tag: https://github.com/caddyserver/xcaddy/releases/download/v0.4.2/xcaddy_linux_amd64.tar.gz

# Instead of `curl -fsSL`, you may prefer `wget -qO-` (available by default on alpine, while curl is available by default on fedora)
# `--no-same-owner` avoids using the `1001:127` user/group assignment (Github runner defaults I think?)
# Extracts only the binary to `/usr/local/bin`
curl -fsSL https://github.com/caddyserver/xcaddy/releases/latest/download/xcaddy_linux_amd64.tar.gz | tar -xz -C /usr/local/bin --no-same-owner xcaddy

A README example could be simpler (assuming most common asset for this demographic would be linux amd64):

curl -fsSL https://github.com/caddyserver/xcaddy/releases/latest/download/xcaddy_linux_amd64.tar.gz | tar -xz xcaddy

I've seen the version "benefit" sometimes encoded into the actual release binary. Either with the asset wrapped in a compressed archive (older releases of hatch) or the binary directly offered (jaq), and others that archived a top-level directory with the version name.

All of those arguably worse since that requires an additional chmod +x or mv to get a friendlier binary name that's executable. Thankfully not a concern here, but definitely taking the perceived benefit a bit too far.

I just realized we're in the xcaddy repo 🤦‍♂️

It would be nice for Caddy too, but I agree with the concern of it causing more friction to switch to than is probably worthwhile. Maybe if Caddy has a 3.0 release some day?

If we converge on yes, it can be added, but we'll have to validate how it impacts the release pipeline.

Sounds like extra complexity than necessary?

I think the general expectation is to improve naming convention of release assets.

The tag in the asset name is redundant in the URL and rarely matters in the asset name itself to the majority of downloaders I'd imagine?
The proposed change would allow the convenience of the latest URL like shown above, no need to browse to releases page to get the versioned URL or modify an outdated one (either locally or from a README example).
Semver wise 0.x.y has each bump of x considered as a breaking change, while x.y.z after the 0.x.y series then has breaking changes as major version. Version bumping breaking changes blindly isn't wise, so any affected automation should be taking that into consideration and check release notes / changelog first to be informed of the release naming convention change.

Your proposal would be a stopgap approach and leaves latest artifacts in prior tagged releases which doesn't do any good? I would discourage that.

mohammed90 commented 1 month ago

Sounds like extra complexity than necessary?

What's the complexity? I'm proposing doing exactly what you're asking. Do you want the version in the asset name or not?

polarathene commented 1 month ago

Do you want the version in the asset name or not?

I expect the following to be valid:

curl -fsSL https://github.com/caddyserver/xcaddy/releases/latest/download/xcaddy_linux_amd64.tar.gz | tar -xz xcaddy
curl -fsSL https://github.com/caddyserver/xcaddy/releases/download/v0.4.2/xcaddy_linux_amd64.tar.gz | tar -xz xcaddy

Drop the tagged version from the asset name (eg: xcaddy_linux_amd64.tar.gz), this is a more common convention.
Note that the URL differs (releases/latest/download/<asset-name> vs releases/download/<tag>/<asset-name>).

We can probably add another build in goreleaser with another ID, e.g. latest, to produce artifacts latest in place of version.

Perhaps I misunderstood what you're suggesting here? There should be no need to produce a separate latest artifacts.

You simply do not include the version in the asset name and publish those assets with the tagged release. After which they can be reached via the usual tagged URL or the latest release published via the latest URL until a new release is published and latest URL shifts to that implicitly.

mohammed90 commented 1 month ago

There should be no need to produce a separate latest artifacts.

What I'm proposing will not include the latest in the artifacts.

polarathene commented 1 month ago

Apologies, I misunderstood your description as something else. We're on the same page then, I'll summarize for clarity 👍

Proposed change only drops the version tag from the name of published assets:

Current: xcaddy_0.4.2_linux_amd64.tar.gz
Proposed: xcaddy_linux_amd64.tar.gz

Thus is just a one line change?

https://github.com/caddyserver/xcaddy/blob/b72e330d5d7439ce848e29ade848760012e9e386/.goreleaser.yml#L72-L83

      {{ .ProjectName }}_
-      {{- .Version }}_
      {{- if eq .Os "darwin" }}mac{{ else }}{{ .Os }}{{ end }}_

we'll have to validate how it impacts the release pipeline.

This is not clear? It should only affect the naming of the archives that the CI publishes to GH releases?

We can probably add another build in goreleaser with another ID, e.g. latest, to produce artifacts latest in place of version.

This is also where I was confused vs the proposal above. Sounded like you were suggesting something different?

EDIT: Oh I see, I think the concern was with how it affects the separate publishing of .deb assets to Cloudsmith. It seems like Cloudsmith does support tags (but the CI action may not?), so that clarifies the concerns you raised a bit better :)

Tags was supported in the action at one point but then reverted. For now there it can be done via the --tags option but note this caveat with --tags version:latest.

francislavoie commented 1 month ago

Thus is just a one line change?

This would be a breaking change for anyone who already has automation to pull the asset with the version in the filename (e.g. the Docker builder image, or anyone else). That's my concern with this.

And also I don't want to do any change that would encourage not pinning to a specific version. It's really trivial to grab the correct file with a regexp or something. I don't see that much value in this change.

polarathene commented 1 month ago

This would be a breaking change for anyone who already has automation to pull the asset with the version in the filename (e.g. the Docker builder image, or anyone else). That's my concern with this.

This was covered already?

It's not breaking in the sense that any pre-existing release is unaffected. A failure will occur when the version is explicitly bumped.
x.y.z / 0.x.y are permitted to breaking changes for x. One should not blindly bump versions subject to breaking changes without checking changelog, that's the point of semver.

If anyone was automating a process that pulls multiple releases, then yes such a change will introduce some friction for that scenario. Feel free to delay until 1.0.0 if that makes you more comfortable?

And also I don't want to do any change that would encourage not pinning to a specific version.

This is akin to :latest with DockerHub, even official Caddy docs don't version pin when pulling the image. The xcaddy README relies on advising packages as "latest" offered from those package managers rather than explicit versions, and there's also the advice:

go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest

Using the latest tag is convenience and generally well understood why it's bad, I wouldn't say you're responsible for that. It can be automated regardless if someone is determined for that functionality. I don't see you having an issue with the other advice to do such elsewhere, so why is it a problem with the already implicit latest GH release URL available?

If you're against supporting such more so than the other examples where it's acceptable, then don't update the README to draw attention to it I guess? It's still a valid convenience like any of the others advised.

It's really trivial to grab the correct file with a regexp or something. I don't see that much value in this change.

Could you provide an example? The latest release tag would need to be sourced from somewhere.

If I want a package manager agnostic way to get the binary without building from source, why is better supporting the latest release URL a problem?

I have no interest in doing such in CI where I'll version pin, and I can grab the URL with a few more clicks and type out the command manually, but it'd be nicer to just not think about the version in those scenarios, especially when giving instructions to someone else. There is value, it's a clear improvement for that specific use-case.

caddyserver / xcaddy

Improve published release artifact names #192