search

cosven / cosven.github.io

个人零碎笔记，博客草稿，阅读笔记
10 stars 0 forks source link

构建系统搭建记录 #43

Closed cosven closed 3 years ago

cosven commented 7 years ago
cosven commented 7 years ago

Jenkins 2.0 docker build flow research

流程记录

  1. 创建项目，添加 webhook
  2. 目前是手动创建一个 Flow ?
  3. Flow 运行 Jenkinsfile，Jenkinsfile 可以跑 pylint，buildout 等

怎样用 docker 搭建一个 jenkins-slave

  1. 安装 docker https://docs.docker.com/engine/installation/linux/debian/
##### ssh docker
sudo ...
docker ps
docker exec -i -t 6a718e6e3a04 /bin/bash

docker exec -u 0 -i -t 6a718e6e3a04 /bin/bash  # login as root

##### install docker
apt-get install apt-transport-https ca-certificates
apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
apt-get install docker-engine
service docker start
docker run hello-world  # test install

##### docker-compose 配置更新时
docker-compose up --build
docker 跑不起来

ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

问题原因：user 所在组不对。使用 sudo 即可。

git 仓库代码都没有 pull 下来

Jenkinsfile 里面没有 git credentialsId: credentialsId, url: url

build 一个 docker 镜像出来

Docker in Docker

参考链接:

Dood

问题

Docker-compose 是在什么时候把 jenkins-cosven 这个命令生成的 ？

Dockerfile 语法

Q & A

jenkins-python 为啥要本地把它跑起来？
跑一个 buildout 为啥会失败？mkdir 失败？
build 不会自动停止？
cosven commented 7 years ago

Jenkins Pipeline

Why Pipeline?

旧式 freestyle job 支持简单地持续集成，允许玩家按照顺序定义一些任务。

pipelines 允许你在一个脚本中定义整个应用生命周期。 pipelines 功能支持持续部署。

  1. Durable: 当你重启 Jenkins Master 时，pipelines 可以存活下来（怎么实现的？）
  2. Pauseable: pipelines 可以停下来等待用户输入...
  3. Versatile: pipelines 支持真实世界的持续部署需求，支持任务并行
  4. Extensible: pipelines 支持用户对它的 DSL 进行扩展...

Pipeline 中的几个专业名词

Step, Node, Stage Master, agent, executor

Pipeline Plugin Context

It is a best practice to develop pipelines as code by storing a Jenkinsfile with pipeline script in your SCM, so that you can apply the same version control and testing to pipelines as you do to your other software

提到一个 Docker Pipeline，其实就是提供一些 docker 操作的接口。感觉用 shell script 代替即可。

....

没啥营养可以查文档的东西...

cosven commented 7 years ago

jenkins blogs

Scaling Jenkins at Jenkins World 2016

https://jenkins.io/blog/2016/08/31/scaling-jenkins-at-jenkins-world/

In many cases when people talk about "scaling Jenkins" they’re talking about "Jenkins as a Service" or "Continuous Delivery as a Service"

An Initiative to Streamline & Automate Jenkin

link: https://www.cloudbees.com/jenkinsops-initiative-streamline-and-automate-jenkins

tags: 没营养

Thinking Inside the Container: A Continuous Delivery Story

link: https://www.cloudbees.com/sites/default/files/2016-jenkins-world-thinking_inside_the_container-_a_continuous_delivery_story.pdf

job as code

docker swarm

Docker 跑 Jenkins 遇到的问题
  1. Docker isn't simple.
  2. Container != VM's (没太懂)
    • 不能挂载远程文件系统
    • 无状态
  3. Garbage Collection (docker-gc 工具)
  4. Maintenance/Failure (维护和错误处理)
  5. 怎样 upgrade (使用最新版 jenkins ？)
  6. credentials & security
    • inject
    • env var
    • third-party secret service

Jenkins Inside Google

Discuss work with Jenkins to meet requirements

– Uptime: Quick automated failover across datacenters – Secure master-node communication (without using SSH) – Performs well at scale

Uptime: Master failover at scale
Concerns
Doing failovers with Jenkins

backing up JENKINS_HOME

Doing the failover

secure master-node communication 使用 gRPC 开发了这一种新的 node 模式...

我们并不关注

Performance at scale

node 连接 master 的时候

  1. custom jars. 下载花费时间
  2. 重新计算 jar checksums 花时间. (它们 patch 了一下，缓存计算结果，只计算一次)
  3. queue lock fix （新版本应该没有这问题了）

Recap

How to Do Continuous Delivery with Jenkins Pipeline, Docker and Kubernetes

tag: 没营养

SCALING JENKINS WITH DOCKER: SWARM, KUBERNETES OR MESOS?

scaling jenkins:

  1. 一个 master 上挂更多的 agents

    • 一些插件支持
    • 缺点：master SPOF；处理多个配置，插件版本；一个 master 连接的 slave 总是有限的

  2. 更多的 masters

    • 不同的组织可以自己提供服务，自己运维
    • 缺点：SSO；统一的配置和运维

对比了三种方案在一些特性方面的对比：比如存储啊，对环境依赖啊，目前版本支持否等问题

So, You Want to Build the World’s Biggest Jenkins Cluster

https://www.cloudbees.com/sites/default/files/2016-jenkins-world-soyouwanttobuildtheworldslargestjenkinscluster_final.pdf

大概在讲 怎样评价 jenkins 集群是否大。

自己家的 Jenkins

  1. master failover 方案
  2. slave 可以先手动部署 5 台左右，完全够用了。

官方文档

调度策略

  1. If a project is configured to stick to one computer, that's always honored.
  2. Jenkins tries to build a project on the same computer that it was previously built.

Setting up 100 nodes Jenkins cluster with Docker Swarm

https://blog.hypriot.com/post/setting-up-100-nodes-jenkins-cluster-with-docker-swarm-in-less-than-10-minutes/

这个方案神奇的跳过了 jenkins-slave 需要 secret 和 node name 的事实？

Jenkins Cluster Setup With Docker Containers

https://dzone.com/articles/jenkins-cluster-setup-with-docker-containers

Jenkins Swarm Plugin

How To Manage Jenkins with Rancher on Ubuntu 14.04

https://www.digitalocean.com/community/tutorials/how-to-manage-jenkins-with-rancher-on-ubuntu-14-04

用到 Rancher 这个东西，通过 Jenkins Username 和 Password 启动的。

所以，大概就是使用一个管理 Docker 的东西，再实现一些 API，然后就实现了管理 Jenkins 集群