Investigate circle config.yml for build performance

[ppremk]

While testing noticed that the build step takes a little long before it starts running the testlink step. Will need to dive in the config.yml file steps to see what is going on there before proposing a fix or suggestion.

[ppremk]

Initial findings:

• Free circle CI plans are provided with 1 container

  

• initial .circleci/config.yml file is configured to run in parallel

• this results in only 1 job being run first while the other is being queued since there is only 1 container available in CircleCI

  

• which results in a total combined run time as below

  

• initial test to switch the jobs to run sequentially is documented bellow

  

• next steps: update configuration file in the template branch to match as below according to the various stages where the config files are updated

version: 2.1
executors:
  ci-executor:
    docker:
      - image: circleci/ruby:2.4.1-node-browsers

jobs:
  build:
    executor: ci-executor
    steps:
      - checkout

      - run:
          name: install dependencies
          command: bundle install

      - run:
          name: build the jekyll site
          command: bundle exec jekyll build

      - run:
          name: notify build is finished
          command: echo "The build is finished!"

  testlinks:
    executor: ci-executor
    steps:
      - checkout

      - run:
          name: install dependencies
          command: bundle install

      - run:
          name: build the jekyll site
          command: bundle exec jekyll build

      - run:
          name: run html proofer
          command: bundle exec htmlproofer ./_site --check-html

workflows:
  version: 2.1

  build_and_test:
    jobs:
      - build
      - testlinks:
          requires:
            - build
           # filters:
           # branches:
           #   only: master

[brianamarie]

@ppremk I've gone through the template repository and changed the config file on each branch to have the matching changes based on your above example. I'm going to reimport the course template repository, and try happy path. Hopefully everything will be working, and working faster! 😄

[hectorsector]

40 can also speed up the build, I just worry about adding so much complexity to the config.

[ppremk]

Reducing the jobs to 1 does not have a big difference in the duration taken to execute the build and test

[ppremk]

40 can also speed up the build, I just worry about adding so much complexity to the config.

Thanks @hectorsector i've also tinkered with the caching part. Where it takes too much time is the installing dependencies part where it is at an average 1.10~ minutes

I will look into a docker images which might have all the dependencies installed.. that could be an option.. but we will need to first test it.

I am also 💯 with you on not introducing complexity to the config. 🙂

[ppremk]

@brianamarie the following changes has very slight improvements

• use latest tag for docker image
• update Gemfile to only install what is required. (remove gem github-pages)

[brianamarie]

👋 @ppremk I'm hoping to update the template repository today with the final versions of the config. What's the latest version that we should use?

[ppremk]

👋 @brianamarie here are the latest progress and updates:

• Custom Docker Image is in this repo
• The custom docker image is uploaded to DockerHub in this link
• Upon testing with caching, the performance was not improving because of the Gemfile.lock has the old dependencies which we have compiled into the custom docker image. Thus the option to not include caching here.
• Finalized config file is as follows

version: 2.1
executors:
  ci-executor:
    docker:
      - image: githubtraining/ci-custom:latest

jobs:
  build:
    executor: ci-executor
    steps:
      - checkout

      - run:
          name: build dependencies
          command: bundle install

      - run:
          name: build the jekyll site
          command: bundle exec jekyll build

      - run:
          name: notify build is finished
          command: echo "The build is finished!"

  testlinks:
    executor: ci-executor
    steps:
      - checkout

      - run:
          name: build dependencies
          command: bundle install

      - run:
          name: build the jekyll site
          command: bundle exec jekyll build

      - run:
          name: notify build is finished
          command: echo "The build is finished!"

      - run:
          name: run html proofer
          command: bundle exec htmlproofer ./_site --check-html

      - run:
          name: notify test is finished
          command: echo "The test is completed!"

workflows:
  version: 2.1

  build_and_test:
    jobs:
      - build
      - testlinks:
          requires:
            - build

• This config file is close to the original version by @hectorsector with 2 minor difference

  • it uses an executor tag for pointing to the docker image
  • the jobs are run in sequential and not parallel (because the free circleci plan only provides 1 free container, configuring it to run in parallel is the same as running it in sequence)

• This also means we do not deviate far off from the original config.yml structure and its supporting instructions

• The overall build and test time now is average 36 seconds compared to the previous 1 minute and 43~ seconds

• Here is the result:

  

Feel free to close this issue once you have completed the actions items mentioned in #75 🙂

[brianamarie]

Thank you SO MUCH @ppremk! Build times are so much improved. I'm going to close this now. 🎉 🎉 🎉