learner0810 commented 2 years ago

What would you like to be added:

Why is this needed: karmada is doing application distribution, currently most applications are delivered via helm charts, does karmada have any plans to support distribution charts?

OCM supports installing applications as Helm Chart and loading remote Chart repositories.

lfbear commented 2 years ago

Sure, I knew this is in plan and the helm chart has done.

lfbear commented 2 years ago

learner0810 commented 2 years ago

@lfbear sorry,Maybe my previous expression is not accurate enough. I meant to distribute the helm application to the Member cluster through Karmada, not install Karmada through helm.

lfbear commented 2 years ago

Oh, I see. That seems a good demand, @RainbowMango What do you think

wawa0210 commented 2 years ago

In general scenarios, users can deploy applications in the following three ways

kubernetes yaml
helm charts
CRD bundle

Kubernetes yaml is usually used for the distribution of a single resource, but in real customer scenarios, applications usually include not only one workload (deployment, daemonset, etc.), but also RBAC, configmap, secret, and multiple deployments, etc. At this time, users generally use helm charts to describe the entire application (similar to crd bundle). Does karmada have any special attention to this scenario?

At present, the open source community already has some open source projects similar to helm CRD,such as https://github.com/fluxcd/helm-operator. We can think about how to integrate efficiently

RainbowMango commented 2 years ago

Yes. Karmada will support helm chart propagating, but no clear solution yet. If anyone could help give a proposal would be appreciated.

learner0810 commented 2 years ago

Use an external component, such as helm-Opertaor and install the helm-Opertaor based on the parameters during cluster installation.

The following is an example of karmada distribution of Helm Charts by helm-operator

Install helm-operator in the member cluster

$ helm install helm-operator fluxcd/helm-operator --namespace flux --set helm.versions=v3

Distribute helmreleases.helm.fluxcd.io to member cluster

apiVersion: policy.karmada.io/v1alpha1
kind: ClusterPropagationPolicy
metadata:
  name: crd-propagation
spec:
  resourceSelectors:
    - apiVersion: apiextensions.k8s.io/v1
      kind: CustomResourceDefinition
      name: helmreleases.helm.fluxcd.io
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Distribute this yaml to member cluster

kind: HelmRelease
metadata:
  name: jaeger-operator
  namespace: default
spec:
  chart:
    repository: $url
    name: jaeger-operator
    version: 2.12.3

Then you can see the jaeger-operator deployed to the Member cluster

Karmada built-in support for helm Charts distribution

Which one do you prefer? @RainbowMango

pidb commented 2 years ago

Can we consider this third-party application package management tool (such as helm) for a unified abstraction, because I know that in addition to helm, there are other management tools such as kustomzie. We should minimize the CRD definition inside karmada, They should be included in a generic CRD.

learner0810 commented 2 years ago

tools：flux Download the flux CLI:

 curl -s https://fluxcd.io/install.sh | sudo bash

Install the toolkit controllers in the flux-system namespace:

flux install

tips:

The Flux tool is installed on each cluster,Using Flux Install requires scientific surfing

If Flux is successfully installed, you can see the following POD

[root@10-6-201-150 ~]# kubectl get pod -n flux-system
NAME                                       READY   STATUS    RESTARTS   AGE
helm-controller-55896d6ccf-dlf8b           1/1     Running   0          15d
kustomize-controller-76795877c9-mbrsk      1/1     Running   0          15d
notification-controller-7ccfbfbb98-lpgjl   1/1     Running   0          15d
source-controller-6b8d9cb5cc-7dbcb         1/1     Running   0          15d

helm

Define a HelmRepository source

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
 name: podinfo
spec:
 interval: 1m
 url: https://stefanprodan.github.io/podinfo     
--- 
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
 name: helm-repo
spec:
 resourceSelectors:
   - apiVersion: source.toolkit.fluxcd.io/v1beta1
     kind: HelmRepository
     name: podinfo
 placement:
   clusterAffinity:
     clusterNames:
       - member1
       - member2

Define a HelmRelease source

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
 name: podinfo
spec:
 interval: 5m
 chart:
   spec:
     chart: podinfo
     version: 5.0.3
     sourceRef:
       kind: HelmRepository
       name: podinfo
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
 name: helm-release
spec:
 resourceSelectors:
   - apiVersion: helm.toolkit.fluxcd.io/v2beta1
     kind: HelmRelease
     name: podinfo
 placement:
   clusterAffinity:
     clusterNames:
       - member1
       - member2

Apply those YAML to karma-apiserver

[root@10-6-201-150 helm]# kubectl apply -f ../helm/
helmrelease.helm.toolkit.fluxcd.io/podinfo created
helmrepository.source.toolkit.fluxcd.io/podinfo created
propagationpolicy.policy.karmada.io/helm-release created
propagationpolicy.policy.karmada.io/helm-repo created

Switch to the distributed cluster

[root@10-6-201-150 ~]# kubectl config use-context member2
Switched to context "member2".
[root@10-6-201-150 ~]# kubectl get pod
NAME                      READY   STATUS    RESTARTS   AGE
podinfo-78c475b77-94x54   1/1     Running   0          104s
[root@10-6-201-150 ~]# helm list -A
NAME     NAMESPACE   REVISION    UPDATED                                 STATUS      CHART           APP VERSION
podinfo  default     1           2021-12-21 07:14:50.73460681 +0000 UTC  deployed    podinfo-5.0.3   5.0.3
[root@10-6-201-150 ~]#

kustomize

Define a Git repository source

  apiVersion: source.toolkit.fluxcd.io/v1beta1
  kind: GitRepository
  metadata:
    name: podinfo
  spec:
    interval: 1m
    url: https://github.com/stefanprodan/podinfo
    ref:
      branch: master
  ---
  apiVersion: policy.karmada.io/v1alpha1
  kind: PropagationPolicy
  metadata:
    name: kust-git
  spec:
    resourceSelectors:
      - apiVersion: source.toolkit.fluxcd.io/v1beta1
        kind: GitRepository
        name: podinfo
    placement:
      clusterAffinity:
        clusterNames:
          - member1
          - member2

Define a kustomization

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
 name: podinfo-dev
spec:
 interval: 5m
 path: "./deploy/overlays/dev/"
 prune: true
 sourceRef:
   kind: GitRepository
   name: podinfo
 validation: client
 timeout: 80s
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
 name: kust-release
spec:
 resourceSelectors:
   - apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
     kind: Kustomization
     name: podinfo-dev
 placement:
   clusterAffinity:
     clusterNames:
       - member1
       - member2

Apply those YAML to karma-apiserver

[root@10-6-201-150 flux]# kubectl apply -f kust/
gitrepository.source.toolkit.fluxcd.io/podinfo created
kustomization.kustomize.toolkit.fluxcd.io/podinfo-dev created
propagationpolicy.policy.karmada.io/kust-git created
propagationpolicy.policy.karmada.io/kust-release created

Switch to the distributed cluster

[root@10-6-201-150 ~]# kubectl get pod -n dev
NAME                        READY   STATUS    RESTARTS   AGE
backend-69c7655cb-rbtrq     1/1     Running   0          15s
cache-bdff5c8dc-mmnbm       1/1     Running   0          15s
frontend-7f98bf6f85-dw4vq   1/1     Running   0          15s

learner0810 commented 2 years ago

@RainbowMango Please Take A Look.This demo is just basic functionality, I can prepare more demos if you need to learn more about Flux's functionality. Please let me know if you have any questions.

RainbowMango commented 2 years ago

Pretty good after a quick look!!! This is more like an integration solution with fluxcd.io. I like it.

RainbowMango commented 2 years ago

I'll look into it and try it on my side and get back to you. Many thanks.

RainbowMango commented 2 years ago

I think we can add a document at docs/work-with-flux.md. The structure would be:

Overview: description about flux
Prerequisites: installing flux.
Propagating Helm by Karmada
Propagating Kustomize by Karmada

How do you think @learner0810 ? And would like to do it?

learner0810 commented 2 years ago

I think we can add a document at docs/work-with-flux.md. The structure would be:

Overview: description about flux

Prerequisites: installing flux.

Propagating Helm by Karmada

Propagating Kustomize by Karmada

How do you think @learner0810 ? And would like to do it?

I agree with the idea. I would be more than happy to complete this document /assign

RainbowMango commented 2 years ago

Thanks, hard to say this is the only way we support helm chart propagation, but it's a very very good start!

RainbowMango commented 2 years ago

Hi @learner0810 Since we are going to cut the new release by the end of this week, so I'm asking when this document will be ready?

learner0810 commented 2 years ago

Hi @learner0810 Since we are going to cut the new release by the end of this week, so I'm asking when this document will be ready?

I am very sorry. I've been very busy lately, so the document is not ready yet, can you put it in the next version?

Poor12 commented 1 year ago

Really interested with flux. Since @learner0810 is busy, I will help with this workguide.

learner0810 commented 1 year ago

Really interested with flux. Since @learner0810 is busy, I will help with this workguide.

I am very sorry that I am writing this work guide. You can look at other issues and say sorry again

Poor12 commented 1 year ago

It doesn't matter. Looking Forward to your contribution.

Poor12 commented 1 year ago

kindly ping @learner0810

RainbowMango commented 1 year ago

Hi @learner0810 Are you still working on this?

learner0810 commented 1 year ago

Hi @learner0810 Are you still working on this?

https://github.com/karmada-io/karmada/pull/1881

I see that someone has mentioned the distribution of helm PR, this document is not needed, I closed this issue, what do you think?

RainbowMango commented 1 year ago

1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.

This issue demonstrates how to integrate flux with Karmada, that's a useful option too. So, we still need a tutorial guide based on your demo above. Feel free to let me know if you don't have enough time to do it.

learner0810 commented 1 year ago

1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.

This issue demonstrates how to integrate flux with Karmada, that's a useful option too. So, we still need a tutorial guide based on your demo above. Feel free to let me know if you don't have enough time to do it.

I see, thanks. I will finish the document by the weekend.

RainbowMango commented 1 year ago

Thanks /assign @learner0810

We are going to cut a new release by the end of this week(probably this Saturday), hope this document could be included in this release.

karmada-io / karmada

karmada will support distribution of helm afterwards？ #861

1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.

1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.