Add rfc to allow pre-release dependencies

c0d1ngm0nk3y commented 1 year ago

Summary

Each buildpack has a list of dependencies that can be provided. This list could also contain pre-releases so that the buildpack can provide early access to some dependencies.

Use Cases

Some dependencies (e.g. sap-machine) provide pre-releases for their releases. By allowing the user to use them via buildpacks, the upcoming version could already be validated.

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[ ] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[ ] I have added an integration test, if necessary.
[x] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

dmikusa commented 1 year ago

I could be missing something, but I'm not aware of a rule that prevents the inclusion of prerelease dependencies. In fact, I can think of a couple of buildpacks off the top of my head that do include prerelease software. Maven is one (I think Tomee has or had some pre-release deps too). We added support for Maven 4, which is in milestone status.

I think we generally don't include prerelease deps just because Paketo buildpacks target "production workloads" and because it creates extra work to track and merge prereleases which often move quickly, but my $0.02 is that if it benefits the buildpack and users, add them.

I don't think we need an RFC for this, unless others feel strongly about it. Again, just my $0.02 but I would say include them like regular dependencies. If we want to standardize around a "preprelase" metadata flag, I'm fine with that, but in general I think it's up to the buildpack to a.) default to an appropriate version (probably not a prerelease) and b.) users to know which version they are selecting.

c0d1ngm0nk3y commented 1 year ago

I could be missing something...

Fair point. I think we phrased the rfc poorly because for major releases, it already works like you pointed out. The use case we have in mind is rather a pre-release for an existing major.

Example:

The user needs java 19 and there are 2 dependencies available: 19.0.2 and 19.0.3-rc3. So the use case would be to enable the user to use 19.0.2 in some pipeline, but to have another one using 19.0.3-rc3 to already validate coming changes.

We "only" had java (sap-machine to be more specific) in mind, but there is no real reason to limit it to java.

We started a RFC for that because it would most likely involve some change to central components (e.g. libpak). When downloading a component, a pre-release should only be picked if the user explicitly ask to include pre-releases. Otherwise 19.0.2 would be picked in that example.

You are right, with major versions, that is a completely different story. If the user wants java 20, it is clear that a pre-release would be picked.

I will rework the rfc and hopefully clarify things.

dmikusa commented 1 year ago

We started a RFC for that because it would most likely involve some change to central components (e.g. libpak). When downloading a component, a pre-release should only be picked if the user explicitly ask to include pre-releases. Otherwise 19.0.2 would be picked in that example.

This would be something to try. Right now, my understanding is that it uses semver to pick the "latest" version. I think that should exclude prereleases unless opted into. At any rate, it would be interesting to see if changes are required or if we can just lean on semver to do this for us. I'd rather not create more convention and metadata if semver can handle things in a reasonable way.

c0d1ngm0nk3y commented 1 year ago

I'd rather not create more convention and metadata if semver can handle things in a reasonable way.

I totally agree. That is what semver is for anyway. But I checked all buildpacks contributing an OpenJDK and most of them have some kind of pre-releases, but NONE is using semver to signal this.

When actually putting the dependency into the buildpack.toml, it would be easy to add this property since this information is available in github releases or the foojay api.

I think that should exclude prereleases unless opted into.

I think so, too. I thought of a environment variable to allow pre-releases to be found.

dmikusa commented 1 year ago

That is what semver is for anyway. But I checked all buildpacks contributing an OpenJDK and most of them have some kind of pre-releases, but NONE is using semver to signal this.

I'm not surprised by that. Let's rectify that though and use semver to do this. Even if we're not now, semver should be capable of doing it. So let's fix up pipeline-builder and buildpack.toml's to include prerelease information where appropriate.

As usual, the hardest part will probably be for things that do not comply with semver & having to stuff those versions into a semver compliant version string.

c0d1ngm0nk3y commented 1 year ago

We had a closer look and it seems easier to have a separate buildpack to deal with that instead of coming up with a generic solution that works for everyone. It is probably "only" relevant for "OpenJDKs" since they might break their users with minor/patch versions when removing ciphers. But addressing this for the specific buildpack we are using, should be rather easy. Hence I will close this RFC.

paketo-buildpacks / rfcs