can we automate Deploy code in repo in a namespace with the same name as the branch

nishit93-hub commented 2 years ago

Hi All,

Can we automate this Deploy code in repo in a namespace with the same name as the branch

vavdoshka commented 2 years ago

@nishit93-hub the easiest example could be

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example-approot
spec:
  generators:
  - scmProvider:
      cloneProtocol: https
      gitlab:
        group: "acme/myrepo"
        allBranches: true
        tokenRef:
          secretName: gitlab-token
          key: token
  template:
    metadata:
      name: {{ repository }}.{{ branch }}
    spec:
      project: default
      source:
        repoURL: {{ url }}
        targetRevision: {{ branch }}
        path: "helm-charts"
      destination:
        server: https://kubernetes.default.svc
        namespace: {{ branch }}

and this can be customized further to have also some filtering on branch naming:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example-approot
spec:
  generators:
  - scmProvider:
      cloneProtocol: https
      gitlab:
        group: "acme/myrepo"
        allBranches: true
        tokenRef:
          secretName: gitlab-token
          key: token
      filters:
        - branchMatch: branch1
        - branchMatch: branch2
  template:
    metadata:
      name: {{ repository }}.{{ branch }}
    spec:
      project: default
      source:
        repoURL: {{ url }}
        targetRevision: {{ branch }}
        path: "helm-charts"
      destination:
        server: https://kubernetes.default.svc
        namespace: {{ branch }}

check more on SCM provider note that ApplicationSet itself should be deployed in the same namespace where ArgoCD is.

nishit93-hub commented 2 years ago

@vavdoshka , Hi Thanks for sharing.

Will these {{ repository }}.{{ branch }} {{ url }} {{ branch }} gets automatically updated once developer creates new branch and commit some changes.?

vavdoshka commented 2 years ago

@nishit93-hub yes, ArgoCD ApplicationSet Controller checks the repository periodically and would submit as many ArgoCD applications as there are branches in the repo, it also will remove the application from ArgoCD if it won't find a correspondent branch.

However, the actual deployment of K8s manifests is controlled by ArgoCD itself, so if you also want the created Applications to deploy manifests automatically you need to slightly adjust the Application template.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example-approot
spec:
  generators:
  - scmProvider:
      cloneProtocol: https
      gitlab:
        group: "acme/myrepo"
        allBranches: true
        tokenRef:
          secretName: gitlab-token
          key: token
  template:
    metadata:
      name: {{ repository }}.{{ branch }}
    spec:
      project: default
      source:
        repoURL: {{ url }}
        targetRevision: {{ branch }}
        path: "helm-charts"
      destination:
        server: https://kubernetes.default.svc
        namespace: {{ branch }}
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
          - PruneLast=true
        automated: 
          prune: true
          allowEmpty: true
          selfHeal: true

In this case, you will get the application created once a branch is created, then ArgoCD will deploy the manifests automatically, and then if there will be any new commits in the same branch ArgoCD will deploy/remove manifests to match with the desired state as well.

nishit93-hub commented 2 years ago

@vavdoshka Hi,

This is my yaml file

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example-approot
  namespace: argocd
spec:
  generators:
  - scmProvider:
      cloneProtocol: https
      gitlab:
        group: "gitops-argo/gitops-dev"
        allBranches: true
        tokenRef:
          secretName: gitops-argo
          key: gitops-argo
  template:
    metadata:
      name: '{{ repository }}'
    spec:
      project: git-repo
      source:
        repoURL: '{{ url }}'
        targetRevision: '{{ branch }}'
        path: wordpress
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{ branch }}'
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
          - PruneLast=true
        automated: 
          prune: true
          allowEmpty: true
          selfHeal: true

group: "gitops-argo/gitops-dev"

gitops-argo Groupname gitops-dev Repo Name

Currently, I have a single branch i.e master. I am trying to deploy this yaml file from the terminal using kubectl apply -f application.yaml command.

However, these are errors I am getting in ApplicationSet controller logs

time="2021-11-26T10:29:10Z" level=error msg="error generating params" error="error listing repos: error listing projects for gitops-argo/gitops-dev: GET https://gitlab.com/api/v4/groups/gitops-argo/gitops-dev/projects: 401 {message: 401 Unauthorized}" generator="&{0xc00057d630 <nil>}"
ERROR   controller-runtime.manager.controller.applicationset    Reconciler error    {"reconciler group": "argoproj.io", "reconciler kind": "ApplicationSet", "name": "example-approot", "namespace": "argocd", "error": "error listing repos: error listing projects for gitops-argo/gitops-dev: GET https://gitlab.com/api/v4/groups/gitops-argo/gitops-dev/projects: 401 {message: 401 Unauthorized}"}

I have added the gitlab token as secret in kubernetes cluster under namespace where argocd is setup. Do i need to explicitly add any details on argo CI?

vavdoshka commented 2 years ago

@nishit93-hub,

the error you got could mean just one thing - the ArgoCD Application Set Controller did find the token but the provided token is not authorized to read data in gitops-argo/gitops-dev. It should be a "Personal Access Token" with read_api access.

Also

  template:
    metadata:
      name: '{{ repository }}'

Since you plan to have more than 1 branch the name should include the branch name cause the names of the ArgoCD applications should be unique.

nishit93-hub commented 2 years ago

@vavdoshka , Yes I was using Deploy tokens and now I used "Personal Access Token". Now Argocd is able to access.

Yes with name: '{{ repository }}' Argocd was not able to deploy multiple branches. I corrected that also.

One thing more I want to ask is, do I need to explicitly add Git URL from Argocd UI? Current, I have to add Git Url from the Argocd dashboard otherwise it throws me an error even though I am providing "Personal Access Token"

vavdoshka commented 2 years ago

@nishit93-hub ,

Yes so with "Personal Access Token" ArgoCD Application Controller can reach out to your repo, it creates the ArgoCD application and from that point, ArgoCD starts its control loop on the application created, and yes it needs to know how to authenticate to the Git URL provided in Application Definition. So there are several ways to configure this in ArgoCD:

manually add repo through UI with url/user/password.
deploy the repository secret with the url and user/password (nothing has to be done in UI) using repository-credentials with label argocd.argoproj.io/secret-type: repository Examples
deploy the repository template secret with the url and user/password (nothing has to be done in UI) using repository-credentials with label argocd.argoproj.io/secret-type: repo-creds which allows reusing a token issued on Group level to access multiple repositories with the same group prefix. That is one token for many repositories at the same time.
create "template secret" manually through UI as well.

BTW the protocol could be HTTPS/SSH, the user/password type of authentication implies HTTPS obviously.

nishit93-hub commented 2 years ago

@vavdoshka, Thanks for the information and all help. I used repository template secret and It worked.

nishit93-hub commented 2 years ago

@vavdoshka Hi, I need your help again with the Argocd image updater. I posted the issue on their Github repo but didn't get any answer there.

Here is the link to the issue. https://github.com/argoproj-labs/argocd-image-updater/issues/308.

Could you please help me in this.

nishit93-hub commented 2 years ago

@vavdoshka Can we use SCM provider with Git SCM ( bare repository)? I am not using Gitlab or Github. I am hosting my own Git server. Or we can use the SCM provider with Gitlab and Github only?

vavdoshka commented 2 years ago

Hello @nishit93-hub , I believe only GitLab and GitHub are supported as of today unfortunatelly

nishit93-hub commented 2 years ago

@vavdoshka Ok, thanks Are there any way I can create a namespace same as branch name with any other Argo Generator.

argoproj / applicationset

can we automate Deploy code in repo in a namespace with the same name as the branch #420