search

kubernetes-sigs / image-builder

Tools for building Kubernetes disk images
https://image-builder.sigs.k8s.io/
Apache License 2.0
377 stars 383 forks source link

Investigate implications of new Packer licensing #1246

Open mboersma opened 1 year ago

mboersma commented 1 year ago

HashiCorp recently changed the license on some of their open source projects to BSL 1.1. This may or may not present legal issues to the use of Packer in the image-builder project.

Let's communicate with the CNCF to see if there's an official position on the new license, and determine whether there are implications for image-builder.

image-builder currently uses Packer v1.9.1 and is updating to v1.9.2, the last version under the MPL. The project will stay pinned there until we have a resolution to this issue.

https://github.com/kubernetes-sigs/image-builder/blob/d2a2af9a6c4da053220e07259b2cdeb49736348d/images/capi/hack/ensure-packer.sh#L23-L24

See also:

AverageMarcus commented 1 year ago

Some points of note:

mboersma commented 1 year ago

/cc @abhay-krishna

jsturtevant commented 1 year ago

thread in the cncf on this topic: https://github.com/cncf/foundation/issues/617

AverageMarcus commented 1 year ago

Recommendations from CNCF: https://github.com/cncf/foundation/blob/main/source-available-recommendations.md

We've already pinned the version but thats a short term solution at best. I don't believe their is an alternative available to Packer that we could switch to, at least not without substantial work needing to be done.

AverageMarcus commented 1 year ago

As discussed in this slack thread: https://cloud-native.slack.com/archives/C0MP69YF4/p1692111612303149?thread_ts=1692108926.999039&cid=C0MP69YF4

We should submit a request for license exception from the CNCF Governing Board (not sure how to do that yet) just to be on the safe side. It's possible we won't need the exception but its best to check to be sure.

If no one else gets to it first I'll try and get the request in Thurs/Fri this week.

BenTheElder commented 1 year ago

We should submit a request for license exception from the CNCF Governing Board (not sure how to do that yet) just to be on the safe side.

From https://github.com/cncf/foundation/blob/main/source-available-recommendations.md:

In either case, while a project's maintainers may request a license exception from the CNCF Governing Board, it is extremely unlikely that one will be granted for the use of a dependency under a source available, non-open source license.

The CNCF has granted exceptions in the past for hashi libraries etc being under MPL, but that is still an "open source" license rather than "source available". I don't think they're likely to approve one here, the CNCF approved license list (no exceptions necessary) are all relatively free use license (ASLv2, MIT, ... https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md) and BUSL is pretty much the opposite.

That said, to file a request you file an issue at https://github.com/cncf/foundation/issues such as https://github.com/cncf/foundation/issues/187

lentzi90 commented 1 year ago

Has anyone looked at potential alternatives to packer? I was not aware of any but did a quick search and found linuxkit. Any ideas if it could replace packer? (Looks like they used to build OS images for Kubernetes with it in this project but it hasn't been updated in 5 years.)

mboersma commented 1 year ago

See also kubernetes-sigs/cluster-api#9181.

AverageMarcus commented 1 year ago

Opened exception request: https://github.com/cncf/foundation/issues/625

BenTheElder commented 1 year ago

Linuxkit is used by docker desktop (see adopters list) which ships kubernetes.

It seems like switching would be a lot of work though.

AverageMarcus commented 1 year ago

It seems like switching would be a lot of work though.

Exactly this. Any alternative that's not api-compatible with Packer would need some serious consideration and input from many users before a decision is made.

mboersma commented 1 year ago

It appears that Packer 1.9.x patches will keep the MPL 2.0 license, although BUSL is already in the main branch. See #1269.

cpboyd commented 11 months ago

The FAQ notes:

Security fixes will be backported under MPL 2.0 through December 31, 2023.

I'm wondering if towards the end of the year there might be more efforts to fork projects like packer similarly to OpenTofu

BenTheElder commented 11 months ago

I think if there were going to be a similar effort for projects other than terraform there would've been some indication of that by now.

justinsb commented 10 months ago

We did have alternative approaches in this project that did not require packer, though they were removed in https://github.com/kubernetes-sigs/image-builder/pull/1175 because they were not well maintained (I was one of the guilty parties!)

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

AverageMarcus commented 7 months ago

/remove-lifecycle stale

lentzi90 commented 4 months ago

The exception request has been rejected: https://github.com/cncf/foundation/issues/625#issuecomment-2127628513

AverageMarcus commented 4 months ago

Yeah. Quite disappointing. 😞

I'm talking with the other maintainers to decide how we proceed. Once we have something we'll update.

vincepri commented 4 months ago

👋 Following this discussion, one point we should clarify: are we bundling Packer today at all and redistributing it?

From what I can recall, for image-builder Packer is used as the primary mean to create artifacts, but it shouldn't be shipped as an artifact itself. While the license exception makes sense it was closed, I wonder if using Packer as a tool, and not embedding it in any form categorizes as fair use.

Not a lawyer, but would like to get some clarification if the above is also out of scope.

@BenTheElder do you know any project using almost-closed-source dependencies in a similar way?

AverageMarcus commented 4 months ago

Yes, we are packaging it and distribute it in the container image that we build and provider for others to use. As far as I understand this counts.

Myself and the other maintainers have been discussing this and have decided that for right now we're not going to be making any changes. We're still pinned at the latest version that uses the old license. This is obviously not a long term solution but it works for now and gives us a chance to not rush in to anything.

There has been some rumours that the IBM purchase of HashiCorp could possibly lead to them reverting the license change back to a real open source license. The likelihood of this is admittedly very low I think but it would be ideal if that is the outcome so we're hoping we can "buy some time" until that is either confirmed or not.

Failing that, the most likely course of action would be to remove Packer from the container image (and possibly not provide a container image at all) and have end users need to bring their own Packer binary that would then be compliant with the license. This isn't ideal as it impacts a lot of our users who would then need to build their own images and it would cause problems for testing and automated CI/CD building of images.

For completeness - replacing Packer with an alternative is almost certainly not going to happen. Rewriting image-builder to use an alternative (if there even was such a tool) or building something custom would be a very large development undertaking and not one we have the resources for. As we don't even have consistency across all providers and OSs in image-builder it would also be incredibly difficult to build something new that didn't break something for some use case. We also discussed the possibility of forking Packer with the open source license like OpenTofu but similarly we don't have the resources to manage that and it seems no one else in the community are interested in taking this on.

I hope that clears some things up. None of this is "set in stone" and we're trying to be flexible but realistic where possible. 😄

mboersma commented 4 months ago

replacing Packer with an alternative is almost certainly not going to happen

I agree with all of what @AverageMarcus says above. In particular, I don't think it's likely we could create a fully functional replacement tool that would support all of the use cases represented by the 180 build- targets in the Makefile. Instead I can imagine two three scenarios:

The first scenario still implies a fair amount of work making sure the fork is supported and up-to-date, and that we somehow can be made aware of security issues in upstream Packer and are able to fix them without running afoul of Hashicorp licensing.

The second scenario would involve something like changing the build-azure-sig-azurelinux-3 target to call the confusingly named Azure Image Builder service (for example) with new JSON or Bicep scripts to produce an image. This could help us move forward, but would be kind of a free-for-all that implies increased support from each CAPI provider, which may not be realistic. It's also not really a community, open source approach.

The third scenario could work, but raises the bar of entry to use image-builder. We also would need to get CNCF legal to agree that was adequate.

vincepri commented 4 months ago

Thanks for the context folks. One question: why are we distributing a container image with packer bundled within? Can we stop publishing it and instead provide guidance and documentation for the intended use?

AverageMarcus commented 4 months ago

Not including Packer in the container image would mean the container isn't actually usable as Packer is required for image-builder to function. We're considering this but it would impact our end users that rely on this image to make use of image-builder so we're reluctant to do so until needed.

Once way I think we might be able to get around this (though I'm not 100% sure if this avoids the license or not) is if we have the dependencies fetched at runtime when launching the container image. This could possibly have implications for users that run it in more locked down environments and could cause the image to stop working if an external dependency isn't available at runtime.

BenTheElder commented 4 months ago

I would avoid any sort of runtime dependency that does not meet CNCF licensing requirements, regardless of distribution or linkage.

So:

(e.g. we fetch shellcheck in kubernetes CI, but users of the Kubernetes release are not expected to use this, and we do NOT host it)

but the ultimate authority on this is CNCF / LF legal.

There has been some rumours that the IBM purchase of HashiCorp could possibly lead to them reverting the license change back to a real open source license. The likelihood of this is admittedly very low I think but it would be ideal if that is the outcome so we're hoping we can "buy some time" until that is either confirmed or not.

We're not depending on this elsewhere in Kubernetes and I wouldn't bet on it, given no intention has been announced. It also may be quite some time before the deal closes even if there were clear intent. ("end of 2024" is the target per the IBM announcement)

Forking pre-BUSL packer could be acceptable but sounds difficult to sustain and probably should be discussed as part of a larger effort under the CNCF or similar.

AverageMarcus commented 4 months ago

Maybe I'm misunderstanding @BenTheElder but it sounds like you're suggesting that we should stop our reliance on Packer completely, is that correct?

BenTheElder commented 4 months ago

I don't think that's the only valid option but I also don't think

Once way I think we might be able to get around this (though I'm not 100% sure if this avoids the license or not) is if we have the dependencies fetched at runtime when launching the container image.

Is a workable approach.

The CNCF recommends freezing at the current version as a short term approach but I think either an exception or a fork will be necessary for long term: https://github.com/cncf/foundation/blob/main/source-available-recommendations.md#recommendations

If there aren't sufficient resources for a fork (or switching tools), you could request an exception (see doc above) and maybe also seek clarification if the "we download it at runtime" approach is acceptable.

AverageMarcus commented 4 months ago

If there aren't sufficient resources for a fork (or switching tools), you could request an exception

We already tried that and it was rejected. 😔 See https://github.com/cncf/foundation/issues/625#issuecomment-2127628513

fabriziopandini commented 4 months ago

I tried to give my 2 cents here

TL;DR; I think the usage the project is doing of packer should not be classified a runtime dependency, because it can lead to confusion.

AFAIK, the project is using packer as a build tool, and packer is not included at all in any of the images (OVA, ami etc) which are the ultimate output of the project

The "commodity docker image" which is publiished as a release artifact, instead is just a commodity to avoid to install dependency locally, not the core output of the project; I understand that this is not ideal, but in case this can help in addressing the issue, IMO the image builder project can consider stopping publishing those "commodity images". Users can always install the required tools locally, or eventually build the "commodity image" locally if they prefer.

BenTheElder commented 4 months ago

Yes that sounds reasonable to me, if the docker image is just used at development time we can stop pre-installing packer when a new [BUSL] version is needed for patch reasons (... and start pre-installing it again if the license changes [back]).

It looks like we're only dodging a ~10-14MB download. Contributors in an airgapped environment can import the binaries alongside the container image. See e.g. the shellcheck scripts in k/k that will install in CI if the binary isn't available / the right version.

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

AverageMarcus commented 1 month ago

/remove-lifecycle stale