search

cncf / foundation

☁️♮🏛 This repo contains several documents related to the operation of the CNCF. File non-technical issues related to CNCF here.
https://cncf.io
Other
553 stars 543 forks source link

Investigate MPL -> BUSL Changes/Impact #617

Closed jeefy closed 3 months ago

jeefy commented 1 year ago

We may have some projects that may be impacted by a license change.

We should investigate the impact on our projects and provide guidance if they are impacted.

Relevant Links: https://www.hashicorp.com/blog/hashicorp-adopts-business-source-license https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md https://github.com/cncf/foundation/blob/main/agpl-recommendations.md https://github.com/cncf/foundation/issues/187 (License exception for hashicorp/terraform projects)

EDIT: Let's keep this a data-gathering Issue only please 😄 See https://github.com/cncf/foundation/issues/617#issuecomment-1678235192 for context.

EDIT EDIT: We've published some initial guidelines here: https://github.com/cncf/foundation/blob/main/source-available-recommendations.md

dims commented 1 year ago

Kubernetes Infra (sig-k8s-infra) has a lot of usages of terraform: https://github.com/search?q=repo%3Akubernetes%2Fk8s.io%20terraform&type=code

Kubernetes image-builder subproject of CAPI uses packer to build AMI(s): https://cs.k8s.io/?q=packer&i=nope&files=&excludeFiles=&repos=kubernetes-sigs/image-builder

Some good news though, Kubernetes used to vendor libraries from hashicorp under MPL for a long time in its history, but over time we started pruning them a while ago, the last of which went in here: https://github.com/kubernetes/kubernetes/pull/103548

And we have tools to prevent regressions to the vendored depdenencies: https://github.com/kubernetes/kubernetes/blob/master/hack/unwanted-dependencies.json#L30

Initial slack discussion: https://kubernetes.slack.com/archives/C5P3FE08M/p1691699636105219

rikatz commented 1 year ago

From a quick search on Vagrant usage:

Sig-windows-dev-tools rely on Vagrant to build the environment https://github.com/kubernetes-sigs/sig-windows-dev-tools

Kubespray (github.com/kubernetes-sigs/kubespray) offers a way to bootstrap using Vagrant

IIUC from the license, dev workflow licensing will not be changed and both tools uses Vagrant for development and not to offer production services, but it is worth checking as some cloud provider may be using at least kubespray internally and this may impact them

BenTheElder commented 1 year ago

While kubernetes core doesn't depend on any hashicorp libraries, plenty of subprojects do. https://cs.k8s.io/?q=%22github.com%2Fhashicorp&i=nope&files=&excludeFiles=&repos=

From a quick scan, I think these are all MPL things that remain MPL for now.

EDIT: We also have some vagrant usage in https://github.com/kubernetes-sigs/kind CI, but nothing critical and we can probably move to lima, we just need to non-interactively boot a cgroupsv2 enabled VM and ssh install/test docker/podman/kind. The kubernetes-sigs/image-builder project is probably the most immediate concern.

vielmetti commented 1 year ago

The Register has a story at https://www.theregister.com/2023/08/11/hashicorp_bsl_licence/ (in the inimitable El Reg style).

jayunit100 commented 1 year ago

Sig windows dev tools uses vagrant

amye commented 1 year ago

Sig windows dev tools uses vagrant

Slightly more details would be helpful!

dims commented 1 year ago

Looking through the https://cs.k8s.io/?q=hashicorp%5C%2F&i=nope&files=go.mod&excludeFiles=&repos= and the exceptions list from https://github.com/cncf/foundation/tree/main/license-exceptions the grand total of 24 repos that seem to get vendored

hashicorp/consul/api
hashicorp/errwrap
hashicorp/go-cleanhttp
hashicorp/go-getter
hashicorp/go-hclog
hashicorp/go-immutable-radix
hashicorp/go-msgpack
hashicorp/go-multierror
hashicorp/go-plugin
hashicorp/go-retryablehttp
hashicorp/go-rootcerts
hashicorp/go-safetemp
hashicorp/go-secure-stdlib
hashicorp/go-sockaddr
hashicorp/go-uuid
hashicorp/go-version
hashicorp/golang-lru
hashicorp/hcl
hashicorp/memberlist
hashicorp/raft
hashicorp/raft-boltdb
hashicorp/serf
hashicorp/vault
hashicorp/yamux
yurishkuro commented 1 year ago

Jaeger backend (https://github.com/jaegertracing/jaeger) uses two Hashicorp libraries:

It is my understanding that libraries are not subject to MPL -> BSL change, but we're watching those repos anyway. We also have a plan to phase out hashicorp/go-plugin (https://github.com/jaegertracing/jaeger/issues/4647).

aaguiarz commented 1 year ago

OpenFGA (https://github.com/openfga/openfga) uses:

OpenFGA's CLI (https://github.com/openfga/cli) uses

Our understanding is that those projects are libraries that are not subject to MPL -> BSL change.

AkihiroSuda commented 1 year ago
AkihiroSuda commented 1 year ago

Lima has a template for Nomad, but we are going to ditch it away

terrytangyuan commented 1 year ago

Argo has one direct and multiple indirect dependencies on HashiCorp projects. My understand is that those dependencies are not subject to MPL -> BSL change. We are tracking those closely at https://github.com/argoproj/argoproj/issues/236.

ckotzbauer commented 1 year ago

Kured uses one library indirectly, I opened https://github.com/kubereboot/kured/issues/817

stefanprodan commented 1 year ago

@dims I think CNCF needs to replace hashicorp/vault with hashicorp/vault/api in the license exceptions, only the API package remains MPL, while the rest is now BUSL 1.1.

justincormack commented 1 year ago

The repos that have changed licenses are below (note as Stefan says, there may be parts that are not relicensed in these repos)

hashicorp/terraform hashicorp/consul hashicorp/vault hashicorp/vagrant hashicorp/nomad hashicorp/packer hashicorp/waypoint hashicorp/boundary hashicorp/vault-csi-provider hashicorp/vault-secrets-operator

All the general Go libraries etc are unchanged.

Sub parts that remain MPL include (not exhaustive check) hashicorp/consul/api hashicorp/vault/api hashicorp/vault/sdk

stefanprodan commented 1 year ago

For the Flux project we are tracking the HashiCorp license change impact here https://github.com/fluxcd/flux2/issues/4156.

While evaluating our usage of HashiCorp Go packages and software products, two questions have been raised:

❓ We need to decide what do to with the Flux Terraform Provider, if CNCF doesn't add the Terraform Plugin SDK (MPL licensed) to the exceptions list we may be forced to stop offering an official Terraform Provider for Flux.

❓ We need to decide what do to with the various end-to-end tests that rely on Terraform for infrastructure bootstrap. We've invested tremendous time in developing automated e2e and conformance tests for Flux 2.0 GA. I hope we can keep using Terraform internally as we don't ship any HashiCorp software with Flux, we only use this software in GitHub Actions Workflows.

jayunit100 commented 1 year ago

... anyways, so re: sig-windows-dev-tools...

JorTurFer commented 1 year ago

Hello, In KEDA (keda.sh) we support HashiCorp Vault as secrets source (we just read values from there as a client), due to it, we use these deps:

github.com/hashicorp/vault/api
github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-cleanhttp v0.5.2 // indirect 
github.com/hashicorp/go-hclog v1.3.0 // indirect 
github.com/hashicorp/go-multierror v1.1.1 // indirect 
github.com/hashicorp/go-retryablehttp v0.7.2 // indirect 
github.com/hashicorp/go-rootcerts v1.0.2 // indirect 
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.7 // indirect 
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect 
github.com/hashicorp/go-sockaddr v1.0.2 // indirect 
github.com/hashicorp/go-uuid v1.0.3 // indirect 
github.com/hashicorp/hcl v1.0.0 // indirect

We also deploy a HashiCorp Vault during e2e test to test the integration (we use helm chart for it). We only use it locally within the testing cluster and we remove it after the e2e test.

For managing e2e test infrastructure we use terraform as well. We manage the infra from its own repo and terraform is executed via GH Action (using an Azure Blob Storage as backend).

I think that we aren't affected because KEDA doesn't provide any service that compits with hashicorp products, so 3rd parties who offer KEDA as service should be safe, but it'd be nice if we could confirm this point.

sftim commented 1 year ago

Using Vagrant is allowed by the new licence, so long as either:

I am of course not a lawyer

ahrtr commented 1 year ago

etcd has only one indirect dependency on library github.com/hashicorp/golang-lru, it seems not subject to the license change based on https://www.hashicorp.com/license-faq

yongtang commented 1 year ago

coredns has indirect dependencies on multiple hashicorp projects through github.com/openzipkin/zipkin-go and gopkg.in/DataDog/dd-trace-go.v1:

github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/errwrap@v1.1.0
github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/go-multierror@v1.1.1
github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/go-uuid@v1.0.3
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/consul/api@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/vault/api@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/vault/sdk@v0.1.14-0.20200519221838-e0cfd64bc267
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/errwrap@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-cleanhttp@v0.5.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-hclog@v0.16.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-immutable-radix@v1.3.1
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-multierror@v1.1.1
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-retryablehttp@v0.6.6
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-rootcerts@v1.0.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-sockaddr@v1.0.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/golang-lru@v0.5.4
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/hcl@v1.0.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/memberlist@v0.1.6
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/serf@v0.8.6
MikeZappa87 commented 1 year ago

Hashicorp pulls in go-cni for nomad. Can we position our license to say they can no longer use it? Or larger to say you cannot use the CNI at all?

dims commented 1 year ago

Hashicorp pulls in go-cni for nomad. Can we position our license to say they can no longer use it? Or larger to say you cannot use the CNI at all?

@MikeZappa87 please let's not go there.

bryantbiggs commented 1 year ago

coredns has indirect dependencies on multiple hashicorp projects through github.com/openzipkin/zipkin-go and gopkg.in/DataDog/dd-trace-go.v1:

github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/errwrap@v1.1.0
github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/go-multierror@v1.1.1
github.com/openzipkin/zipkin-go@v0.4.1 github.com/hashicorp/go-uuid@v1.0.3
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/consul/api@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/vault/api@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/vault/sdk@v0.1.14-0.20200519221838-e0cfd64bc267
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/errwrap@v1.1.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-cleanhttp@v0.5.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-hclog@v0.16.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-immutable-radix@v1.3.1
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-multierror@v1.1.1
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-retryablehttp@v0.6.6
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-rootcerts@v1.0.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/go-sockaddr@v1.0.2
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/golang-lru@v0.5.4
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/hcl@v1.0.0
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/memberlist@v0.1.6
gopkg.in/DataDog/dd-trace-go.v1@v1.53.0 github.com/hashicorp/serf@v0.8.6

These are all fine per https://www.hashicorp.com/license-faq#What-did-HashiCorp-announce-today-(Aug-10)

HashiCorp APIs, SDKs, and almost all other libraries will remain MPL 2.0.

abvaidya commented 1 year ago

Athenz provides a terraform provider - https://github.com/AthenZ/terraform-provider-athenz and uses following libraries -

github.com/hashicorp/go-cty v1.4.1-0.20200723130312-85980079f637
    github.com/hashicorp/terraform-plugin-sdk/v2 v2.27.0

    github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-checkpoint v0.5.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-plugin v1.4.10 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/go-version v1.6.0 // indirect
github.com/hashicorp/hc-install v0.5.2 // indirect
github.com/hashicorp/hcl/v2 v2.17.0 // indirect
github.com/hashicorp/logutils v1.0.0 // indirect
github.com/hashicorp/terraform-exec v0.18.1 // indirect
github.com/hashicorp/terraform-json v0.17.0 // indirect
github.com/hashicorp/terraform-plugin-go v0.18.0 // indirect
github.com/hashicorp/terraform-plugin-log v0.9.0 // indirect
github.com/hashicorp/terraform-registry-address v0.2.2 // indirect
github.com/hashicorp/terraform-svchost v0.1.1 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
AkihiroSuda commented 1 year ago

@jeefy

Investigate MPL -> BSL Changes/Impact

Could you consider s/BSL/BUSL/ in the issue title?

"BSL" stands for "Boost Software License" (OSI-approved, very permissive license) in SPDX:

amye commented 1 year ago

@jeefy

Investigate MPL -> BSL Changes/Impact

Could you consider s/BSL/BUSL/ in the issue title?

"BSL" stands for "Boost Software License" (OSI-approved, very permissive license) in SPDX:

Done

jeefy commented 1 year ago

My bad, thanks!

fedordikarev commented 1 year ago

if it can be useful: Dashboard with overview of modules and their licences: https://fedordikarev.grafana.net/dashboard/snapshot/AVkaQa2He4cgKOfXisVPZGTF2c5oS6eY (2023-08-13 12:25 UTC: updated column widths for better UX)

and here is code how to get and reproduce that data: https://github.com/fedordikarev/misc/tree/master/cncf_hashicorp

dims commented 1 year ago

thanks @fedordikarev that's really helpful

mattray commented 1 year ago

OpenCost uses several of their Go libraries, which they've indicated they won't change. From the go.mod

    github.com/hashicorp/go-multierror v1.0.0

and

        github.com/hashicorp/errwrap v1.0.0 // indirect
    github.com/hashicorp/hcl v1.0.0 // indirect
sagikazarmark commented 1 year ago

Dex has a pending PR that introduces github.com/hashicorp/vault/api which continues to be released as MPL.

I reached out to HashiCorp regarding another open source project I maintain (Bank-Vaults; https://bank-vaults.dev/). It has competing elements with HashiCorp offerings (Bank-Vaults operates and configures Vault). Their position is that as long as Bank-Vaults is not commercialized, we can continue supporting new Vault releases under BUSL.

Saim-Safdar commented 1 year ago

@amye In Paralus, we did a quick scan of our Paralus repos, we do not use any hashicorp libraries directly and the only one used indirectly is hashicorp/hcl which is still under MPL license. So afaik, we are not impacted by this change, is it correct or do we need to update anything?

antoninbas commented 1 year ago

For Project Antrea:

Will the CNCF provide guidance about whether it is ok to keep using Terraform / Vagrant in development / CI workflows?

Edit: added links for reference

amye commented 1 year ago

For Project Antrea:

  • We use some of the hashicorp Go libraries, which remain licensed under MPL
  • We provide some Terraform scripts as a reference for users and developers (for testing)
  • We provide a Vagrantfile, once again that can be used by developers for testing

Will the CNCF provide guidance about whether it is ok to keep using Terraform / Vagrant in development / CI workflows?

We're just gathering data right now to investigate.

amye commented 1 year ago

Link to me where you're using that? It'll be helpful to track.

trask commented 1 year ago

For OpenTelemetry we are investigating in https://github.com/open-telemetry/community/issues/1643

kapilt commented 1 year ago

For cloud custodian project, we use several of the hashicorp go libraries (still MPL licensed) to parse / evaluate terraform (not tf repo directly), we extensively use terraform for functional testing.

[update] from go.mod / go.sum the exact HashiCorp MPL libraries in use are

github.com/hashicorp/go-cleanhttp
github.com/hashicorp/go-getter
github.com/hashicorp/go-safetemp
github.com/hashicorp/go-uuid
github.com/hashicorp/go-version
github.com/hashicorp/golang-lru 
github.com/hashicorp/hcl/v2
jamietanna commented 1 year ago

Is there a handy list of all GitHub organisations that are under the CNCF banner? I may be able to help with collating some data based on all the GitHub orgs, but may need some pointers, aside from:

parispittman commented 1 year ago

Is there a handy list of all GitHub organisations that are under the CNCF banner? I may be able to help with collating some data based on all the GitHub orgs, but may need some pointers, aside from:

  • cncf

  • kubernetes

  • kubernetes-sig

  • cloud-custodian

  • open-telemetry

https://github.com/cncf/devstats/blob/master/scripts/all/repo_groups.sql

this could be a good start

amye commented 1 year ago

Is there a handy list of all GitHub organisations that are under the CNCF banner? I may be able to help with collating some data based on all the GitHub orgs, but may need some pointers, aside from:

  • cncf
  • kubernetes
  • kubernetes-sig
  • cloud-custodian
  • open-telemetry

Right now, we're just looking to hear from the projects with their specific uses!

jeefy commented 1 year ago

Good call @parispittman :)

Look for the org_login field that's set. Most projects only have a single Org, then you have Kubernetes:

org_login in (
      'kubernetes', 'kubernetes-client', 'kubernetes-incubator', 'kubernetes-csi',
      'kubernetes-graveyard', 'kubernetes-incubator-retired', 'kubernetes-sig-testing',
      'kubernetes-providers', 'kubernetes-addons', 'kubernetes-retired',
      'kubernetes-extensions', 'kubernetes-federation', 'kubernetes-security',
      'kubernetes-sidecars', 'kubernetes-tools', 'kubernetes-test', 'kubernetes-sigs'
    )
BenTheElder commented 1 year ago

[...] whether it is ok to keep using Terraform / Vagrant in development / CI workflows?

My concern: Vendors that help staff projects may not be able to replicate these workflows on BUSL licensed versions going forward. If that is the case, I don't think we should be depending on them even for workflows.

E.G. if we have a CI test environment with vagrant, ideally everyone participating can replicate it, but perhaps not going forward ...

The CNCF sandbox project lima may be a replacement in that specific case.

I'm not a lawyer, it's possible this isn't the case, hopefully we can get some clarification.

Another concern: The ecosystem around these may not be as healthy going forward, so they may be a technical liability.

bryantbiggs commented 1 year ago

cc @solutiongeek

jeefy commented 1 year ago

Let's keep this thread primarily around the gathering of info (meaning: references to potentially-impacted code) and not bikeshedding ourselves into oblivion (as we are often want to do 🤣).

This is totally my fault for not scoping this properly, so I apologize.

I'd suggest project-specific discussions occur within the respective communities, and if people want a separate meta issue I (or y'all) can create one in cncf/foundation. But for the sake of all (primarily @amye and the LF folks involved) this issue should stay a data-gathering one. <3

philpennock commented 1 year ago

For NATS, and using go.mod / go.sum as my primary reference:

I don't believe there's much else, for the open source side of things. Separately from that, and not directly something the CNCF will care about, Synadia (the company doing most NATS development) does extensively use Packer and Terraform for building and running our services. That only impacts NATS-the-project insofar as changing Synadia's use would necessarily divert development effort from NATS.

savitharaghunathan commented 1 year ago

For Konveyor project, we are using the following 

* github.com/hashicorp/errwrap v1.1.0 
* github.com/hashicorp/go-multierror v1.1.1
* github.com/hashicorp/go-version v1.6.0
* github.com/hashicorp/hcl v1.0.0
* Vagrant in development env and setup docs
oshoval commented 1 year ago

Created this if it helps https://github.com/kubevirt/cluster-network-addons-operator/pull/1595 It whitelists the HashiCorp modules that stayed MPL (or non restrictive such as MIT etc), Only the ones we are using are included, not the full list, but it is easy to extend of course. It can also be used to find out the ones that are not whitelisted and categorize them manually. This PR assumes those will stay MPL, and fail on the others, in case someone adds a module which is not whitelisted

amye commented 1 year ago

We've published some guidelines here that may be helpful to projects reviewing this issue: https://github.com/cncf/foundation/blob/main/source-available-recommendations.md

lizrice commented 1 year ago

Cilium uses the following: 

    github.com/hashicorp/consul/api 
    github.com/hashicorp/errwrap 
    github.com/hashicorp/go-cleanhttp 
    github.com/hashicorp/go-hclog 
    github.com/hashicorp/go-immutable-radix 
    github.com/hashicorp/go-immutable-radix/v2 
    github.com/hashicorp/go-memdb 
    github.com/hashicorp/go-multierror 
    github.com/hashicorp/go-rootcerts 
    github.com/hashicorp/golang-lru 
    github.com/hashicorp/golang-lru/v2 
    github.com/hashicorp/hcl 
    github.com/hashicorp/memberlist 
    github.com/hashicorp/packer-plugin-sdk 
    github.com/hashicorp/serf