Open WangShuXian6 opened 5 years ago
https://docs.gitlab.com.cn GitLab 中文文档 GitLab CI 是 GitLab 为了提升其在软件开发工程中作用,完善 DevOPS 理念所加入的 CI/CD 基础功能。可以便捷的融入软件开发环节中。通过 GitLab CI 可以定义完善的 CI/CD Pipeline。
- GitLab CI 是默认包含在 GitLab 中的,我们的代码使用 GitLab 进行托管,这样可以很容易的进行集成
- GitLab CI 的前端界面比较美观,容易被人接受
- 包含实时构建日志,容易追踪
- 采用 C/S 的架构,可方面的进行横向扩展,性能上不会有影响
- 使用 YAML 进行配置,任何人都可以很方便的使用。
Pipeline 相当于一个构建任务,里面可以包含多个流程,如依赖安装、编译、测试、部署等。 任何提交或者 Merge Request 的合并都可以触发 Pipeline
Stage 表示构建的阶段,即上面提到的流程.
- 所有 Stages 按顺序执行,即当一个 Stage 完成后,下一个 Stage 才会开始
- 任一 Stage 失败,后面的 Stages 将永不会执行,Pipeline 失败
- 只有当所有 Stages 完成后,Pipeline 才会成功
Job 是 Stage 中的任务.
- 相同 Stage 中的 Jobs 会并行执行
- 任一 Job 失败,那么 Stage 失败,Pipeline 失败
- 相同 Stage 中的 Jobs 都执行成功时,该 Stage 成功
Runner 是任务的实际执行者, 可以在 MacOS/Linux/Windows 等系统上运行。使用 golang 进行开发。 同时也可部署在 k8s 上
将 runner 注册为一个容器
docker run --rm -t -i -v /path/to/config:/etc/gitlab-runner --name gitlab-runner gitlab/gitlab-runner register \ --executor "docker" \ --docker-image alpine:3 \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-runner" \ --tag-list "dev" \ --run-untagged \ --locked="true"
在物理机上的注册方式
sudo gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --executor "docker" \ --docker-image alpine:3 \ --description "docker-runner" \ --tag-list "docker,aws" \ --run-untagged \ --locked="false" \
runner 的类型
- Shared - Runner runs jobs from all unassigned projects
- Group - Runner runs jobs from all unassigned projects in its group
- Specific - Runner runs jobs from assigned projects
- Locked - Runner cannot be assigned to other projects
- Paused - Runner will not receive any new jobs
https://www.youtube.com/watch?v=eBOvoy6ThB0
gitlab mac 安装教程: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/docker/README.md
gitlab 其他平台安装教程: https://about.gitlab.com/install/
docker 安装gitlab 命令:
sudo docker run --detach \ --hostname gitlab.example.com \ --publish 443:443 --publish 80:80 --publish 22:22 \ --name gitlab \ --restart always \ --volume /Users/wangshuxian/gitlab/config:/etc/gitlab \ --volume /Users/wangshuxian/gitlab/logs:/var/log/gitlab \ --volume /Users/wangshuxian/gitlab/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest
也可使用企业版 gitlab/gitlab-ee:latest hostname 可以为内网或本机,万维网地址。 本机地址需在Host映射 127.0.0.1 gitlab.example.com 局域网需在Host映射 局域网ip gitlab.example.com 万维网同上
volume 为本机文件夹,手动创建 wangshuxian 对应系统当前账户文件夹
首次安装指定安装源,以后只使用本机命命 gitlab
首次安装后docker后台配置需要等待较长时间
首次登陆 gitlab.example.com 需要设置管理员root密码
可以继续使用root登陆,或者注册新账户
sudo docker run --detach \
--hostname gitlab.xxx.com \
--publish 4433:443 --publish 8080:80 --publish 2222:22 \
--name gitlab \
--restart always \
--volume ~/gitlab/config:/etc/gitlab \
--volume ~/gitlab/logs:/var/log/gitlab \
--volume ~/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ee:latest
https://docs.gitlab.com/omnibus/settings/nginx.html#redirect-http-requests-to-https
将证书文件上传至gitlab xxx.key xxx.pem
scp -P端口号 本地文件路径 username@服务器ip:目的路径
编辑 gitlab.rb
cd ~/gitlab/config vi gitlab.rb
修改gitlab配置文件
external_url 'https://xxx.xxx.com/' nginx['redirect_http_to_https'] = true nginx['ssl_certificate'] = "/etc/gitlab/xxx.pem" nginx['ssl_certificate_key'] = "/etc/gitlab/xxx.key" gitlab_rails['time_zone'] = 'Asia/Shanghai'
进入容器交互命令行
docker exec -it gitlab bash
使配置生效
gitlab-ctl reconfigure
Both GitLab CE and EE are in Docker Hub:
The GitLab Docker images are monolithic images of GitLab running all the necessary services on a single container.
In the following examples we are using the image of GitLab CE. To use GitLab EE
instead of GitLab CE, replace the image name to gitlab/gitlab-ee:latest
.
If you want to use the latest RC image, use gitlab/gitlab-ce:rc
or
gitlab/gitlab-ee:rc
for GitLab CE and GitLab EE respectively.
The GitLab Docker images can be run in multiple ways:
Docker installation is required, see the official installation docs.
Note: Using a native Docker install instead of Docker Toolbox is recommended in order to use the persisted volumes
Warning: We do not officially support running on Docker for Windows. There are known issues with volume permissions, and potentially other unknown issues. If you are trying to run on Docker for Windows, please see our getting help page for links to community resources (IRC, forum, etc) to seek help from other users.
Run the image:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
This will download and start a GitLab CE container and publish ports needed to
access SSH, HTTP and HTTPS. All GitLab data will be stored as subdirectories of
/srv/gitlab/
. The container will automatically restart
after a system reboot.
You can now login to the web interface as explained in After starting a container.
If you are on SELinux then run this instead:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab:Z \
--volume /srv/gitlab/logs:/var/log/gitlab:Z \
--volume /srv/gitlab/data:/var/opt/gitlab:Z \
gitlab/gitlab-ce:latest
This will ensure that the Docker process has enough permissions to create the config files in the mounted volumes.
The GitLab container uses host mounted volumes to store persistent data:
Local location | Container location | Usage |
---|---|---|
/srv/gitlab/data |
/var/opt/gitlab |
For storing application data |
/srv/gitlab/logs |
/var/log/gitlab |
For storing logs |
/srv/gitlab/config |
/etc/gitlab |
For storing the GitLab configuration files |
You can fine tune these directories to meet your requirements.
This container uses the official Omnibus GitLab package, so all configuration
is done in the unique configuration file /etc/gitlab/gitlab.rb
.
To access GitLab's configuration file, you can start a shell session in the context of a running container. This will allow you to browse all directories and use your favorite text editor:
sudo docker exec -it gitlab /bin/bash
You can also just edit /etc/gitlab/gitlab.rb
:
sudo docker exec -it gitlab vi /etc/gitlab/gitlab.rb
Once you open /etc/gitlab/gitlab.rb
make sure to set the external_url
to
point to a valid URL.
To receive e-mails from GitLab you have to configure the SMTP settings because the GitLab Docker image doesn't have an SMTP server installed.
You may also be interested in Enabling HTTPS.
After you make all the changes you want, you will need to restart the container in order to reconfigure GitLab:
sudo docker restart gitlab
Note: GitLab will reconfigure itself whenever the container starts.
For more options about configuring GitLab please check the Omnibus GitLab documentation.
You can pre-configure the GitLab Docker image by adding the environment
variable GITLAB_OMNIBUS_CONFIG
to docker run command. This variable can
contain any gitlab.rb
setting and will be evaluated before loading the
container's gitlab.rb
file. That way you can easily configure GitLab's
external URL, make any database configuration or any other option from the
Omnibus GitLab template.
_Note: The settings contained in GITLAB_OMNIBUS_CONFIG
will not be written to the gitlab.rb
configuration file, they're evaluated on load._
Here's an example that sets the external URL and enables LFS while starting the container:
sudo docker run --detach \
--hostname gitlab.example.com \
--env GITLAB_OMNIBUS_CONFIG="external_url 'http://my.domain.com/'; gitlab_rails['lfs_enabled'] = true;" \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
Note that every time you execute a docker run
command, you need to provide
the GITLAB_OMNIBUS_CONFIG
option. The content of GITLAB_OMNIBUS_CONFIG
is
not preserved between subsequent runs.
There are also a limited number of environment variables to configure GitLab. They are documented in the environment variables section of the GitLab documentation.
After starting a container you can visit http://localhost/ or http://192.168.59.103 if you use boot2docker. It might take a while before the Docker container starts to respond to queries.
Note: The initialization process may take a long time. You can track this
process with the command sudo docker logs -f gitlab
The very first time you visit GitLab, you will be asked to set up the admin
password. After you change it, you can login with username root
and the
password you set up.
To upgrade GitLab to a new version you have to:
Stop the running container:
sudo docker stop gitlab
Remove existing container:
sudo docker rm gitlab
Pull the new image:
sudo docker pull gitlab/gitlab-ce:latest
Create the container once again with previously specified options:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
On the first run, GitLab will reconfigure and update itself.
We provide tagged versions of GitLab Docker images.
To see all available tags check:
To use a specific tagged version, replace gitlab/gitlab-ce:latest
with
the GitLab version you want to run, for example gitlab/gitlab-ce:8.4.3
.
You can make Docker to use your IP address and forward all traffic to the
GitLab CE container by modifying the --publish
flag.
To expose GitLab CE on IP 1.1.1.1:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 1.1.1.1:443:443 \
--publish 1.1.1.1:80:80 \
--publish 1.1.1.1:22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
You can then access your GitLab instance at http://1.1.1.1/
and https://1.1.1.1/
.
GitLab will occupy some ports inside the container.
If you want to use a different host port than 80
(HTTP) or 443
(HTTPS),
you need to add a separate --publish
directive to the docker run
command.
Do NOT use any of the above ports to replace the defaults 80
or 443
inside
the container, otherwise there will be conflicts.
For example, to expose the web interface on port 8929
, and the SSH service on
port 2289
, use the following docker run
command:
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 8929:80 --publish 2289:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
NOTE: Note:
The format for publishing ports is hostPort:containerPort
. Read more in
Docker's documentation about exposing incoming ports.
You then need to appropriately configure gitlab.rb
:
Set external_url
:
# For HTTP
external_url "http://gitlab.example.com:8929"
or
# For HTTPS (notice the https)
external_url "https://gitlab.example.com:8929"
For more information see the NGINX documentation.
Set gitlab_shell_ssh_port
:
gitlab_rails['gitlab_shell_ssh_port'] = 2289
Following the above example you will be able to reach GitLab from your
web browser under <hostIP>:8929
and push using SSH under the port 2289
.
A docker-compose.yml
example that uses different ports can be found in the
docker-compose section.
Read container logs:
sudo docker logs gitlab
Enter running container:
sudo docker exec -it gitlab /bin/bash
From within the container you can administer the GitLab container as you would normally administer an Omnibus installation
With Docker compose you can easily configure, install, and upgrade your Docker-based GitLab installation.
Install Docker Compose
Create a docker-compose.yml
file (or download an example):
web:
image: 'gitlab/gitlab-ce:latest'
restart: always
hostname: 'gitlab.example.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.example.com'
# Add any other gitlab.rb configuration here, each on its own line
ports:
- '80:80'
- '443:443'
- '22:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
Make sure you are in the same directory as docker-compose.yml
and run
docker-compose up -d
to start GitLab
Read "Pre-configure Docker container" to see
how the GITLAB_OMNIBUS_CONFIG
variable works.
Below is another docker-compose.yml
example with GitLab running on a custom
HTTP and SSH port. Notice how the GITLAB_OMNIBUS_CONFIG
variables match the
ports
section:
web:
image: 'gitlab/gitlab-ce:latest'
restart: always
hostname: 'gitlab.example.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://gitlab.example.com:9090'
gitlab_rails['gitlab_shell_ssh_port'] = 2224
ports:
- '9090:9090'
- '2224:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
This is the same as using --publish 9090:9090 --publish 2224:22
.
Provided you installed GitLab using docker-compose,
all you have to do is run docker-compose pull
and docker-compose up -d
to
download a new release and upgrade your GitLab instance.
With Docker swarm you can easily configure and deploy your Docker-based GitLab installation in a swarm cluster.
In swarm mode you can leverage Docker secrets and Docker configs to efficiently and securely deploy your GitLab instance. Secrets can be used to securely pass your initial root password without exposing it as an environment variable. Configs can help you to keep your GitLab image as generic as possible.
Here's an example that deploys GitLab with four runners as a stack, using secrets and configs:
Create a docker-compose.yml
file:
version: "3.6"
services:
gitlab:
image: gitlab/gitlab-ce:latest
ports:
- "22:22"
- "80:80"
- "443:443"
volumes:
- /srv/gitlab/data:/var/opt/gitlab
- /srv/gitlab/logs:/var/log/gitlab
- /srv/gitlab/config:/etc/gitlab
environment:
GITLAB_OMNIBUS_CONFIG: "from_file('/omnibus_config.rb')"
configs:
- source: gitlab
target: /omnibus_config.rb
secrets:
- gitlab_root_password
gitlab-runner:
image: gitlab/gitlab-runner:alpine
deploy:
mode: replicated
replicas: 4
configs:
gitlab:
file: ./gitlab.rb
secrets:
gitlab_root_password:
file: ./root_password.txt
For simplicity reasons, the network
configuration was omitted.
More information can be found in the official Compose file reference.
Create a gitlab.rb
file:
external_url 'https://my.domain.com/'
gitlab_rails['initial_root_password'] = File.read('/run/secrets/gitlab_root_password')
Create a root_password.txt
file:
MySuperSecretAndSecurePass0rd!
Make sure you are in the same directory as docker-compose.yml
and run:
docker stack deploy --compose-file docker-compose.yml mystack
The GitLab Docker images can also be deployed to various container scheduling platforms.
When updating the Docker image you may encounter an issue where all paths
display the infamous 500 page. If this occurs, try to run
sudo docker restart gitlab
to restart the container and rectify the issue.
When updating from older GitLab Docker images you might encounter permission problems. This happens due to a fact that users in previous images were not preserved correctly. There's script that fixes permissions for all files.
To fix your container, simply execute update-permissions
and restart the
container afterwards:
sudo docker exec gitlab update-permissions
sudo docker restart gitlab
This error occurs when using Docker Toolbox with VirtualBox on Windows or Mac, and making use of Docker volumes. The /c/Users volume is mounted as a VirtualBox Shared Folder, and does not support the all POSIX filesystem features. The directory ownership and permissions cannot be changed without remounting, and GitLab fails.
Our recommendation is to switch to using the native Docker install for your platform, instead of using Docker Toolbox.
If you cannot use the native Docker install (Windows 10 Home Edition, or Windows < 10), then an alternative solution is to setup NFS mounts instead of VirtualBox shares for Docker Toolbox's boot2docker.
If you are using file ACLs on the docker host, the docker
[^1] group requires full access to the volumes in order for GitLab to work.
$ getfacl /srv/gitlab
# file: /srv/gitlab
# owner: XXXX
# group: XXXX
user::rwx
group::rwx
group:docker:rwx
mask::rwx
default:user::rwx
default:group::rwx
default:group:docker:rwx
default:mask::rwx
default:other::r-x
If these are not correct, set them with:
$ sudo setfacl -mR default:group:docker:rwx /srv/gitlab
[^1]: docker
is the default group, if you've changed this, update your commands accordingly.
If your problem is not listed here please see getting help for the support channels.
These docker images are officially supported by GitLab Inc. and should always be up to date.
.gitlab-ci.yml
include: "/.variables.yml"
stages:
#- test
- build
- deploy
cache:
#key: ${CI_COMMIT_REF_NAME}
paths:
- node_modules/
#test_dev:
# image: node:alpine
# stage: test
# only:
# - dev
# tags:
# - shell
# script:
# - npm run test
build_devlop:
#image: node:alpine
stage: build
only:
- develop
tags:
- shell
script:
- wget https://nodejs.org/dist/v10.15.3/node-v10.15.3-linux-x64.tar.xz
- xz -d node-v10.15.3-linux-x64.tar.xz
- tar -xvf node-v10.15.3-linux-x64.tar
- ./node-v10.15.3-linux-x64/bin/npm set registry https://registry.npm.taobao.org # 设置淘宝镜像地址
- ./node-v10.15.3-linux-x64/bin/npm install --progress=false
- ./node-v10.15.3-linux-x64/bin/npm run build:dev
artifacts:
expire_in: 1 week
paths:
- dist
build_master:
#image: node:alpine
stage: build
only:
- master
tags:
- shell
script:
- wget https://nodejs.org/dist/v10.15.3/node-v10.15.3-linux-x64.tar.xz
- xz -d node-v10.15.3-linux-x64.tar.xz
- tar -xvf node-v10.15.3-linux-x64.tar
- ./node-v10.15.3-linux-x64/bin/npm set registry https://registry.npm.taobao.org # 设置淘宝镜像地址
- ./node-v10.15.3-linux-x64/bin/npm install --progress=false
- ./node-v10.15.3-linux-x64/bin/npm run build
artifacts:
expire_in: 1 week
paths:
- dist
deploy_develop:
#image: alpine
stage: deploy
only:
- develop
tags:
- shell
environment:
name: ''
url: ''
script:
- "'"
when: manual
deploy_master:
#image: alpine
stage: deploy
only:
- master
tags:
- shell
environment:
name: ''
url: ''
script:
- "'"
when: manual
config/gitlab-runner/config.toml
concurrent = 4
[[runners]] name = "shell" url = "https://CI/" token = "TOKEN" limit = 2 executor = "shell" builds_dir = "" shell = "bash"
[[runners]] name = "ruby-2.6-docker" url = "https://CI/" token = "TOKEN" limit = 0 executor = "docker" builds_dir = "" [runners.docker] host = "" image = "ruby:2.6" privileged = false disable_cache = false cache_dir = ""
[[runners]] name = "ubuntu-docker-ssh" url = "https://CI/" token = "TOKEN" limit = 0 executor = "docker-ssh" builds_dir = "" [runners.docker] host = "" image = "ubuntu-upstart:14.04" privileged = false disable_entrypoint_overwrite = false disable_cache = false cache_dir = "" [runners.docker.sysctls] "net.ipv4.ip_forward" = "1" [runners.ssh] port = "22" user = "root" password = "docker.io"
[[runners]] name = "production-server" url = "https://CI/" token = "TOKEN" limit = 0 executor = "ssh" builds_dir = "" [runners.ssh] host = "my-production-server" port = "22" user = "root" password = "production-server-password"
##### 部署文件
>```docker-compose.yml```
```yml
version: '3.7'
services:
gitlab-runner-node:
container_name: gitlab-runner-node
image: gitlab/gitlab-runner:alpine
deploy:
mode: replicated
replicas: 4
restart: unless-stopped
privileged: true
# depends_on:
# - gitlab
volumes:
- ./config/gitlab-runner:/etc/gitlab-runner:Z
- /var/run/docker.sock:/var/run/docker.sock
#- /bin/docker:/bin/docker
networks:
- default
# - gitlab
# networks:
# nginx-proxy:
# external: true
su root
sudo gpasswd -a username docker
sudo gpasswd -a ${USER} docker
docker-compose up -d
docker-compose down
项目-seettings-CI/CD-Runners-Specific Runners
public/gitlab-runner-register.sh
每次新注册只需要拷贝一份更新registration_token
值即可#!/bin/sh # Get the registration token from: # http://localhost:8080/root/${project}/settings/ci_cd
registration_token=beDcJB35HCNDsG1v1obR
docker exec -it gitlab-runner-node \ gitlab-runner register \ --non-interactive \ --registration-token ${registration_token} \ --locked=false \ --description docker-stable \ --url https://gitlab.xxx.com:8443/ \ --executor docker \ --docker-image docker:stable \ --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
##### 修复脚本文件权限 后注册
```bash
chmod 777 ./housekeeper/gitlab-runner-register.sh
./housekeeper/gitlab-runner-register.sh
.gitlab-ci.yml
stages: - build - deploy
build: image: node:alpine stage: build script:
#- npm run build
#expire_in: 1 week
#paths:
#- dist/
only:
deploy_staging: image: alpine:latest stage: deploy before_script:
#- apk add --no-cache rsync openssh
script:
#- echo "$SSH_PRIVATE_KEY" >> ~/.ssh/id_dsa
#- chmod 600 ~/.ssh/id_dsa
#- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
#- rsync -rav --delete dist/ "$SERVER_USER_HOST:$SERVER_MASTER_PATH"
only:
持续集成/持续部署