search

imshota / system_development_project_application_1

講義用
0 stars 0 forks source link

継続的インテグレーション #8

Open imshota opened 3 years ago

imshota commented 3 years ago

演習1 CircleCIを使っているOSSを2つ探し、CircleCIの設定を読み、何をしているか調査せよ

作業記録 CircleCI テスト，ビルド，デプロイをカバー

CircleCIの設定は.circleci/config.ymlファイルに(大抵)ある。

OSS CirclCI https://github.com/circleci/circleci-images/blob/staging/.circleci/config.yml

version: 2.1

orbs:
  circle-compare-url: iynere/compare-url@0.4.8

jobs:
  generate_dockerfiles:
    working_directory: ~/repo
    resource_class: small
    docker: [image: gcc:7]
    steps:
      - checkout
      - run:
          name: Generate Dockerfiles
          command: make -j
      - run:
          name: Prepare Artifacts
          command: |
            mkdir /tmp/dockerfiles
            find . -name Dockerfile -type f | grep -v shared | xargs -I{} cp -v --parents {} /tmp/dockerfiles
      - store_artifacts:
          path: /tmp/dockerfiles
          destination: Dockerfiles

      - attach_workspace:
          at: workspace
...

ドメイン名がないので、DockerHubのイメージを使っているっぽい。 dockerfilesのディレクトリを作り、Dockerfileという名前のファイルからshared以外の行をコピーしている。

OSS facebook https://github.com/facebookresearch/detectron2/blob/master/.circleci/config.yml

version: 2.1

# -------------------------------------------------------------------------------------
# Environments to run the jobs in
# -------------------------------------------------------------------------------------
cpu: &cpu
  docker:
    - image: circleci/python:3.6.8-stretch
      auth:
        username: $DOCKERHUB_USERNAME
        password: $DOCKERHUB_TOKEN
  resource_class: medium

gpu: &gpu
  machine:
    image: ubuntu-1604-cuda-10.1:201909-23
  resource_class: gpu.small

windows-cpu: &windows_cpu
  machine:
    resource_class: windows.medium
    image: windows-server-2019-vs2019:stable
    shell: powershell.exe
...

イメージ CircleCIが提供するRuby用イメージ Ubuntu 16.04、docker 18.09.3、docker-compose 1.23.1 windows server 2019

このように複数のイメージをそれぞれで設定することも可能

...
install_python: &install_python
  - run:
      name: Install Python
      working_directory: ~/
      command: |
        pyenv install -s 3.6.8
        pyenv global 3.6.8
        python --version
        which python
        pip install --upgrade pip
...

pythonのインストールを一貫して行っている

imshota commented 3 years ago

演習2 自分のOSSにCircleCIを導入し、Push時にハローワールドを表示させよ

作業記録

参考サイト https://circleci.com/docs/ja/2.0/getting-started/

このyamlファイルだが、 インラインに注意！ ずれていると、エラーが起きる(pythonみたいな感じ)

version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.4.1
    steps:
      - checkout
      - run: echo "hello world"

これで、コミットしてプッシュすると自動的に走る。

  1. 環境のスピンアップ: CircleCI は circleci/ruby:2.4.1 Docker イメージを使用して仮想コンピューティング環境をローンチする。

Spin up environment

Build-agent version 1.0.46069-9c37e851 (2020-11-19T12:12:42+0000)
Docker Engine Version: 18.09.6
Kernel Version: Linux bef285fdeb5d 4.15.0-1077-aws #81-Ubuntu SMP Wed Jun 24 16:48:15 UTC 2020 x86_64 Linux
Starting container circleci/ruby:2.4.1
Warning: No authentication provided, using CircleCI credentials for pulls from Docker Hub.
  image cache not found on this host, downloading circleci/ruby:2.4.1
2.4.1: Pulling from circleci/ruby

ad1a0d33: Already exists 
3158a136: Already exists 
23742a64: Already exists 
5e832c38: Already exists 
736737e7: Already exists 
d7e8047d: Already exists 
41bf1206: Already exists 
c84e5780: Already exists 
f0406b07: Already exists 
e0bd4627: Already exists 
c74cac90: Already exists 
80e1631e: Already exists 
90af60e8: Already exists 
e1231150: Already exists 
e1596557: Already exists 
Digest: sha256:14fd7278f2762c8a93b1de5f1bafa6d5dd282390a52cdc2fcd3207d093b3e0ea
Status: Downloaded newer image for circleci/ruby:2.4.1
  pull stats: N/A
  time to create container: 798ms
  using image circleci/ruby@sha256:14fd7278f2762c8a93b1de5f1bafa6d5dd282390a52cdc2fcd3207d093b3e0ea
Time to upload agent and config: 876.626168ms
Time to start containers: 1.269926314s

Preparing environment variables

Using build environment variables:
  BASH_ENV=/tmp/.bash_env-5fb66c228500a90000d5073e-0-build
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=master
  CIRCLE_BUILD_NUM=9
  CIRCLE_BUILD_URL=https://circleci.com/gh/imshota/my_atcoder/9
  CIRCLE_COMPARE_URL=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  CIRCLE_PREVIOUS_BUILD_NUM=8
  CIRCLE_PROJECT_REPONAME=my_atcoder
  CIRCLE_PROJECT_USERNAME=imshota
  CIRCLE_REPOSITORY_URL=git@github.com:imshota/my_atcoder.git
  CIRCLE_SHA1=279197ea850e6bf1be43715a35f9f1b5aabb6817
  CIRCLE_SHELL_ENV=/tmp/.bash_env-5fb66c228500a90000d5073e-0-build
  CIRCLE_STAGE=build
  CIRCLE_USERNAME=imshota
  CIRCLE_WORKFLOW_ID=c149d892-b040-4194-a63d-c63328285f41
  CIRCLE_WORKFLOW_JOB_ID=25160013-11c9-4ff3-9225-a695f904e363
  CIRCLE_WORKFLOW_UPSTREAM_JOB_IDS=
  CIRCLE_WORKFLOW_WORKSPACE_ID=c149d892-b040-4194-a63d-c63328285f41
  CIRCLE_WORKING_DIRECTORY=~/project

Using environment variables from project settings and/or contexts:
  CIRCLE_JOB=**REDACTED**

The redacted variables listed above will be masked in run step output.
  1. コードのチェック アウト: CircleCI は、GitHub リポジトリを検査し、ステップ 1 でローンチした仮想環境内に「クローン」を作成する。

checkout code

#!/bin/sh
set -e

# Workaround old docker images with incorrect $HOME
# check https://github.com/docker/docker/issues/2968 for details
if [ "${HOME}" = "/" ]
then
  export HOME=$(getent passwd $(id -un) | cut -d: -f6)
fi

echo Using SSH Config Dir $SSH_CONFIG_DIR

mkdir -p $SSH_CONFIG_DIR

echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
bitbucket.org ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAubiN81eDcafrgMeLzaFPsw2kNvEcqTKl/VqLat/MaB33pZy0y3rJZtnqwR2qOOvbwKZYKiEO1O6VqNEBxKvJJelCq0dTXWT5pbO2gDXC6h6QDXCaHo6pOHGPUy+YBaGQRGuSusMEASYiWunYN0vCAI8QaXnWMXNMdFP3jHAJH0eDsoiGnLPBlBp4TNm6rYI74nMzgz3B9IikW4WVK+dc8KZJZWYjAuORU3jc1c/NPskD2ASinf8v3xnfXeukU0sJ5N6m5E8VLjObPEO+mN2t/FZTMZLiFqPWc/ALSqnMnnhwrNi2rbfg/rd/IpL8Le3pSBne8+seeFVBoGqzHM9yXw==
' >> $SSH_CONFIG_DIR/known_hosts

(umask 077; touch $SSH_CONFIG_DIR/id_rsa)
chmod 0600 $SSH_CONFIG_DIR/id_rsa
(cat <<EOF > $SSH_CONFIG_DIR/id_rsa
$CHECKOUT_KEY
EOF
)

export GIT_SSH_COMMAND='ssh -i $SSH_CONFIG_DIR/id_rsa -o UserKnownHostsFile=$SSH_CONFIG_DIR/known_hosts'

# use git+ssh instead of https
git config --global url."ssh://git@github.com".insteadOf "https://github.com" || true
git config --global gc.auto 0 || true

if [ -e /home/circleci/project/.git ]
then
  cd /home/circleci/project
  git remote set-url origin "$CIRCLE_REPOSITORY_URL" || true
else
  mkdir -p /home/circleci/project
  cd /home/circleci/project
  git clone "$CIRCLE_REPOSITORY_URL" .
fi

if [ -n "$CIRCLE_TAG" ]
then
  git fetch --force origin "refs/tags/${CIRCLE_TAG}"
else
  git fetch --force origin 'master:remotes/origin/master'
fi

if [ -n "$CIRCLE_TAG" ]
then
  git reset --hard "$CIRCLE_SHA1"
  git checkout -q "$CIRCLE_TAG"
elif [ -n "$CIRCLE_BRANCH" ]
then
  git reset --hard "$CIRCLE_SHA1"
  git checkout -q -B "$CIRCLE_BRANCH"
fi

git reset --hard "$CIRCLE_SHA1"
Using SSH Config Dir /home/circleci/.ssh
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of known hosts.
remote: Enumerating objects: 51, done.        
remote: Counting objects: 100% (51/51), done.        
remote: Compressing objects: 100% (40/40), done.        
remote: Total 51 (delta 6), reused 16 (delta 0), pack-reused 0        
Receiving objects: 100% (51/51), 10.05 KiB | 0 bytes/s, done.
Resolving deltas: 100% (6/6), done.
Checking connectivity... done.
HEAD is now at 279197e Update config.yml
HEAD is now at 279197e Update config.yml
  1. コマンドの実行

echo "hello world"

#!/bin/bash -eo pipefail
echo "hello world"
hello world
CircleCI received exit code 0
imshota commented 3 years ago

演習3 CircleCIを用いて、自分のOSSのテストを自動で実施せよ

作業記録

yamlファイル

version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.4.1
    steps:
      - checkout
      - run: make ci

実行結果

./ABC/171/test.sh a
make[1]: Entering directory '/home/circleci/project/ABC/171'
g++ -std=c++17 -o b.out b.cpp
g++: error: unrecognized command line option ‘-std=c++17’
makefile:13: recipe for target 'b.out' failed
make[1]: *** [b.out] Error 1
make[1]: Leaving directory '/home/circleci/project/ABC/171'
start testing

オプション -std=c++17 が入っていないらしい・・・

imshota commented 3 years ago

演習4 CircleCIを用いてアーティファクトを自動でアップロードせよ

作業記録

参考ページ https://circleci.com/docs/ja/2.0/artifacts/

参考ページ通り、アーティファクトを自動でアップデートさせた。

version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.4.1
    steps:
      - checkout
      - run:
          name: create artifacts
          command: |
            echo "my artifact file" > /tmp/artifact-1;
            mkdir /tmp/artifacts;
            echo "my artifact files in a dir" > /tmp/artifacts/artifact-2;
      - store_artifacts:
          path: /tmp/artifact-1
          destination: artifact-file
      - run: make ci

ここで、アップロードされていることがわかる https://app.circleci.com/pipelines/github/imshota/my_atcoder/31/workflows/84dc77dc-b785-4dd5-a2a5-accf9490085c/jobs/32/artifacts