search

go-atomci / atomci

🐳 为中国企业而来，以应用为中心的云原生cicd平台
https://go-atomci.github.io/atomci-press/
Apache License 2.0
212 stars 49 forks source link

V2.0 使用argo workflow替换Jenkins 的Proposal #191

Open colynn opened 1 year ago

colynn commented 1 year ago

您有什么需求，是否与某个功能或问题相关? 请描述

因为atomci 1.0版本的pipeline对于jenkins的依赖过多，且灵活性不够，并且Jenkins与代码主程序是是割裂，也经常性出现安全漏洞；

另外一个好的cicd平台肯定是需要大家一起共建才能完成，故v2.0的版本也会引入插件化的理念，让我们每个人均可以定义/贡献自己的流程，期望argo workflow的替换可以成功，之后主要的进程会更新在这个issue内。

你想要的解决方案是什么

如题

#

colynn commented 1 year ago
fanhousanbu commented 1 year ago

argo-workflow是计划怎么部署？由atomci在引入构建机的时候自动部署吗？

colynn commented 1 year ago

argo-workflow是计划怎么部署？由atomci在引入构建机的时候自动部署吗？

计划是先走openAPI的方式先将argo-workflow 引入进来，通过多容器的方式来运行；

colynn commented 1 year ago

Try to use argo-workflow open API

Generate Token

# create role
kubectl create role atomci -n argo  --verb=get,list,update,create,delete --resource=workflows.argoproj.io

# create serviceaccount
kubectl create sa atomci -n argo

# create rolebinding
kubectl create rolebinding atomci-binding -n argo --role=atomci --serviceaccount=argo:atomci

### get token
kubectl -n argo describe sa atomci |grep secrets

# atomci-token-d4zgj get from the above's command
kubectl -n argo get secrets atomci-token-d4zgj -o=jsonpath='{.data.token}' | base64 --decode

Token Usage&Test

ARGO_TOKEN="Bearer $(kubectl get secret jenkins.service-account-token -o=jsonpath='{.data.token}' | base64 --decode)"
echo $ARGO_TOKEN 

curl https://localhost:2746/api/v1/workflows/argo -H "Authorization: $ARGO_TOKEN"
# 200 OK
colynn commented 1 year ago

the mapping of atomci core concept and argo-workflow

AtomCI Argo-workflow Notes
任务模板 Container 类型的template
项目流程 Workflow template
构建部署-创建流水线 submit --from workflowtemplate/ with parameters
流水线 workflow instance

Container-type's template sample

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: compile-go-project
spec:
  entrypoint: compile-go
  templates:
    - name: compile-go
      container:
        image: golang:latest
        command: ["/bin/bash", "-c"]
        args:
          - |
            set -e
            go mod download
            go build -o my-app
        volumeMounts:
          - name: source
            mountPath: /go/src/my-project
          - name: output
            mountPath: /artifacts
      outputs:
        artifacts:
          - name: my-app
            path: /artifacts/my-app
      volumes:
        - name: source
          configMap:
            name: my-project-source
        - name: output
          emptyDir: {}

WorkflowTemplate sample

# filename: ci-template-sample.yaml
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: ci-sample
  annotations:
    workflows.argoproj.io/description: |
      This workflows builds and tests Argo Workflows.

      It demonstrates:
      * Clone/ Build
spec:
  arguments:
    parameters:
      - name: branch
        value: master
  entrypoint: main

  volumes:
    - hostPath:
        path: /tmp/golang
        type: Directory
      name: work

  templates:
    - name: main
      steps:
      - - name: clone
          template: clone
      - - name: build
          template: build
    - name: clone
      container:
        image: golang:1.18
        workingDir: /go/src/github.com/golang/example
        command: [ sh, -euxc ]
        args:
          - |
            git clone -v -b "{{workflow.parameters.branch}}"  --single-branch --depth 1 https://github.com/golang/example.git .

        volumeMounts:
          - mountPath: /go/src/github.com/golang/example
            name: work
            subPath: src
    - name: build
      container:
        image: golang:1.18
        workingDir: /go/src/github.com/golang/example
        command: [ sh, -euxc ] 
        args:
          - |
            cd hello ; go build 
        volumeMounts:
          - mountPath: /go/src/github.com/golang/example
            name: work
            subPath: src 
# create workflowtemplate
argo template create [-n argo]  ci-template-sample.yaml

#  then submit a workflow using this template:
argo submit --from workflowtemplate/ci-sample -p branch="master"
colynn commented 1 year ago

artifacts的使用注意

如果我们想通过artifacts的属性来传递构建物，那么首先需要正确配置Artifact Repository Ref.

You can reduce duplication in your templates by configuring repositories that can be accessed by any workflow. This can also remove sensitive information from your templates.

When you want to use any keyword(eg artifact) , you should remember this link will give you a lot of help.

colynn commented 1 year ago

argo-workflowTemplate in real world

requirements

# create docker-config secret
kubectl create secret generic docker-config -n argo --from-file=/root/.docker/config.json
# create  Minio secret
# TODO: accesskey/secretkey you need change to real.
apiVersion: v1
data:
  accesskey: x
  secretkey: x
kind: Secret
metadata:
  name: s3-credentials
  namespace: argo
type: Opaque

workflow template

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: ci-sample
  annotations:
    workflows.argoproj.io/description: |
      This workflows builds and tests Argo Workflows.

      It demonstrates:
      * Clone/ Build
spec:
  serviceAccountName: argo
  arguments:
    parameters:
      - name: branch
        value: master
  entrypoint: main

  volumes:
    - hostPath:
        path: /tmp/codespace/
        type: Directory
      name: work
    - hostPath:
        path: /var/run/docker.sock
        type: Socket
      name: dockersock
    # Mount the configuration so we can push the image.
    # This should create the /.docker/config.json file.
    - name: docker-config
      secret:
        secretName: docker-config

  templates:
    - name: main
      steps:
      - - name: clone
          template: clone
      - - name: build
          template: build
      - - name: docker-image
          template: image

    - name: clone
      container:
        image: golang:1.18
        workingDir: /go/codespace/src
        command: [ sh, -euxc ]
        args:
          - |
            rm -rf golang-app-demo;  git clone -v -b "{{workflow.parameters.branch}}"  --single-branch https://github.com/go-atomci/golang-app-demo.git 
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src
    - name: build
      container:
        image: golang:1.18
        workingDir: /go/codespace/src
        command: [ sh, -euxc ] 
        args:
          - |
            cd golang-app-demo; go build -o bin/sample  cmd/sample/main.go 
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src 
      outputs:
        artifacts:
          - name: sample-binary
            path: golang-app-demo/bin/sample
            s3:
              endpoint: minio-default.component:9000
              bucket: argo-artifacts
              insecure: true
              key: sample.tgz
              accessKeySecret:
                name: s3-credentials
                key: accesskey
              secretKeySecret:
                name: s3-credentials
                key: secretkey

    - name: image
      container:
        image: alpine:3.13
        workingDir: /go/codespace/src
        command: [ sh, -euxc ]
        args:
          - |
            [ -f docker-19.03.15.tgz ] ||  wget http://pkg.infra.sensetime.com/artifactory/depend/gitlab/docker-19.03.15.tgz ;
            tar --extract --file docker-19.03.15.tgz --strip-components 1 --directory /usr/local/bin/ ;
            docker version ;
            cd golang-app-demo ; 
            docker build -t 10.151.3.75/library/golang-app-demo:latest  -f Dockerfile . ;
            docker push 10.151.3.75/library/golang-app-demo:latest
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src
          - mountPath: /var/run/docker.sock
            name: dockersock
          - name: docker-config
            mountPath: /.docker
        env:
          - name: DOCKER_CONFIG
            value: /.docker

then run it

# create workflowtemplate
argo template create [-n argo]  ci-template-sample.yaml

#  then submit a workflow using this template:
argo submit --from workflowtemplate/ci-sample -p branch="main"
colynn commented 1 year ago

Noice

# List workflows
GET /api/v1/workflows/argo?listOptions.limit=50

# list workflow-template
GET /api/v1/workflow-templates/argo?listOptions.limit=50

# List cron-workflows
GET /api/v1/cron-workflows/argo

### workflow action
# Resubmit workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/resubmit

# suspend workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/suspend

# resume workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/resume

# delete workflow
DELETE /api/v1/workflows/argo/ci-sample-s7c8z

# get workflow
GET /api/v1/workflows/argo/ci-sample-s7c8z

approve

  - name: approve
    suspend: {}

Reference to: https://github.com/argoproj/argo-workflows/blob/master/examples/suspend-template.yaml

colynn commented 1 year ago

We know that if we delete the workflow's pod, then you will not see the log of this workflow, So we need to archive the workflow's pod logs.

Argo-workflow Configuring Archive Logs

Archive logs follow priorities: workflow-controller config (on) > workflow spec (on/off) > template (on/off)

We do not recommend you rely on Argo Workflows to archive logs. Instead, use a conventional Kubernetes logging facility.

https://argoproj.github.io/argo-workflows/configure-archive-logs/

enable archive logs

  1. You need config the Artifact Repository first.

The actual repository used by a workflow is chosen by the following rules:

https://argoproj.github.io/argo-workflows/workflow-controller-configmap/

For AtomCI 's TODO

疑问

%Reply: 当然可以配置多个artifact repository, 如果 workflowTemplate/ workflow/ cluster workflowTemplate 定义了artifact repository 通过声明artifactRepositoryRef则使用其定义，

若均没有定义则使用默认的 artifact repository定义， 若没有默认定义将会失败并返回错误。

Argo key-only artifact

When these are omitted, the bucket/secrets from the configured artifact repository is used. 仅仅定义一个key及path, 其他的信息从artifact repository ref定义中获取。 This should probably be your default if you're using argo-workflow v3.0

colynn commented 1 year ago

服务结构