Add support for creating private catalogs to app platform

rossf7 commented 2 years ago

Is your feature request related to a problem? Please describe.

As a user of app platform I'd like to be able to create a private app catalog for deploying apps that should be kept private.

Currently all app catalogs must be public.

Describe the solution you'd like

Our app catalogs are also Helm repositories which can be secured with HTTP Basic Authentication.

We could add this to app platform by adding the credentials to our Catalog CRs and extending chart-operator to be able to pull charts from private catalogs.

Describe alternatives you've considered

helm-controller supports private helm repositories via the helmrepository CRD.

If in the future we replaced chart-operator with helm-controller we would not need to implement this.

rossf7 commented 2 years ago

@gianfranco-l Here is the roadmap issue. PTAL

gianfranco-l commented 2 years ago

@rossf7 what do you think is the effort here? is there any "quick" win? this info is really key to understand the priority of this

rossf7 commented 2 years ago

@gianfranco-l The effort is large as unfortunately there is no quick win here. I added the effort/l label.

teemow commented 2 years ago

@gianfranco-l this is going to be a big blocker if we want to encourage customers to use the app platform too.

rossf7 commented 2 years ago

@teemow I agree only public catalogs will block many customers from using app platform for their apps.

I discussed with @piontec in our 1:1 and we agreed I'd post my ideas on how to implement this. See next comment.

rossf7 commented 2 years ago

Here is a list of the changes needed to implement.

Private OCI registries should also work if an oci:// URL is used in the catalog CR. Provided helmclient can access private OCI registries.

Catalog CRD

A new .spec.storage.secret block references a secret with the credentials.
No changes are needed to the App CRD.

apiVersion: application.giantswarm.io/v1alpha1
kind: Catalog
metadata:
  name: my-private-catalog
  namespace: org-dev
spec:
  description: 'This catalog is private. '
  logoURL: /images/repo_icons/managed.png
  storage:
    URL: https://example.org/private-catalog/
    secret:
      name: my-private-catalog-auth
      namespace: org-dev
    type: helm
  title: My Private Catalog
---
apiVersion: v1
kind: Secret
metadata:
  name: my-private-catalog-auth
  namespace: org-dev
stringData:
  username: example
  password: 123456

Chart CRD

A new .spec.catalog.secret block is added to the chart CRD.
It references a secret with the creds provided by the user.

apiVersion: application.giantswarm.io/v1alpha1
kind: Chart
metadata:
  name: my-private-app
  namespace: giantswarm
spec:
  catalog:
    secret:
      name: my-private-app-auth
      namespace: giantswarm
  name: my-private-app
  namespace: giantswarm
  tarballURL: https://example.org/private-catalog/my-private-app-0.1.0.tgz
  version: 0.1.0
---
apiVersion: v1
kind: Secret
metadata:
  name: my-private-app-auth
  namespace: giantswarm
stringData:
  username: example
  password: 123456

app-operator

secret resource needs to generate a secret with the creds as well as the current values secret in .spec.config.secret
chart resource needs to set the new fields in the chart CR

chart-operator / helmclient

release resource needs to check if .spec.catalog.secret is set in the chart CR.

helmclient

Use HTTP basic auth to pull chart tarballs with http(s) URLs.
Authenticate with private OCI registries for oci:// URLs.

PG2000 commented 2 years ago

Is their any plan to implement and release it?

gianfranco-l commented 2 years ago

We are planning to work on this within Q4 2022 (but might be worked on it sooner)

luca-rui commented 2 years ago

FYI: We have an additional customer interested in this feature.

gianfranco-l commented 2 years ago

Planning of this has been moved ahead compared to the previous ETA of Q4 2022. We'll try to plan this for first half of 2023

weatherhog commented 2 months ago

@teemow can you tell if this is still a valid request, by our customers?

teemow commented 2 months ago

I'd say definitely. In case they want to use any non open source software the whole app platform becomes useless otherwise.

giantswarm / roadmap