kfctl apply from pricate kfdef on private and enterprize github

[moficodes]

right now the only way we can deploy kubeflow with kfctl is if the manifest is coming from a public repository and the kfdef is either downloaded or is also a public url.

it would be useful if we added a way to download private github files using token and from enterprise github.

[moficodes]

/assign

[arun-vc]

This would be very useful since some component overlays need to be private

[soleares]

If you're trying to add a private github repo to your kfdef in order to specify custom versions of some applications this works now in master:

• Build kfdef from master. It needs this PR: https://github.com/kubeflow/kfctl/pull/247
  • git clone https://github.com/kubeflow/kfctl.git
  • cd kfctl
  • make install
• Append an entry to the repos list for your internal github repo. This assumes that you use SSH to access it.

  repos:
  - name: manifests
  uri: git::https://github.com/kubeflow/manifests/archive/v1.2.0.tar.gz
  - name: my-manifests
  uri: git::git@github.com:myorg/myrepo.git

• If you need to access a specific branch/tag/commit you can add a ref

  repos:
  - name: manifests
  uri: git::https://github.com/kubeflow/manifests/archive/v1.2.0.tar.gz
  - name: my-manifests
  uri: git::git@github.com:myorg/myrepo.git?ref=mybranch

[shalberd]

@pdmack @Tomcli I wonder if adding an environment variable SSL_CERT_DIR or SSL_CERT_FILE to the environment that embeds Kubeflow kfctl

@AlexanderEkdahl you seemed to be the one who initiated the fix for considering http and https proxies a while ago, thank you.

https://github.com/opendatahub-io/opendatahub-operator/commit/16eba4a1eb9bcc241e44046b24b46adb97c629dd

Can you tell from your perspective whether SSL_CERT_DIR and SSL_CERT_FILE, one of the two, are enough to specifiy when making plain http requests with the golang client? That is, are the changes from package x509

https://go.dev/src/crypto/x509/root_unix.go

https://github.com/golang/go/commit/e83bcd95a4a86e4caf2faa78158170d512dd9de5

considered in the requests here

https://github.com/opendatahub-io/opendatahub-operator/blob/master/pkg/kfconfig/types.go#L530

using x509.SystemCertPool implicitly?

Before 2017, one needed to add it manually:

https://stackoverflow.com/questions/38822764/how-to-send-a-https-request-with-a-certificate-golang

keywords: caCertPool.AppendCertsFromPEM and rootCAs argument.

After 2017, it should be there already in x509.loadSystemRoots

If yes, that would accomplish the job of making it possible to trust enterprise PKI CAs. Kubeflow kfctl as well as opendatahub-operator are using net/http for downloading manifest files:

https://github.com/devgrok/kubeflow-kfctl/blob/master/pkg/kfconfig/types.go#L508

Does that imply that SSL_CERT_DIR and SSL_CERT_FILE envs are honored when doing chain of trust validation in go?