search

devcontainers / ci

A GitHub Action and Azure DevOps Task designed to simplify using Dev Containers (https://containers.dev) in CI/CD systems.
MIT License
303 stars 46 forks source link

[Feature] Add an input to skip the "build container" step #225

Open 0xGosu opened 1 year ago

0xGosu commented 1 year ago

In our project, we have already pre-built the DevContainer and published as an image. By skipping this step it would reduce the time to run our pipeline on GitHub actions a lot. Thanks.

stuartleeks commented 1 year ago

Hi @0xGosu - is the build step not using the pre-built image as a cache for the build step?

0xGosu commented 1 year ago

@stuartleeks It is using the pre-built image and rebuild a new image for devcontainer which took around 1-2 minutes. However I'm expecting it to not even run the build step at all as there is no more software need to be installed to the devcontainer image. Below is my devcontainer.json file

// For format details, see https://aka.ms/devcontainer.json
{
  "name": "My example of devcontainer.json",
  // This is a pre-built image contain all tool required
  "image": "org/pre-built-image:latest",
  // Add any feature will cause devcontainer image to be rebuilt
  // "features": {
  // },
  "overrideCommand": false,
  "privileged": true,
  "mounts": [
    // Enable docker dind
    {
      "source": "dind-var-lib-docker-${devcontainerId}",
      "target": "/var/lib/docker",
      "type": "volume"
    }
  ],
  "runArgs": [
    "--network-alias",
    "${devcontainerId}",
    "--label",
    "DEVCONTAINER_REF=${devcontainerId}"
  ],

  // Use 'forwardPorts' to make a list of ports inside the container available locally.
  "forwardPorts": [3000, 5432, 6379],

  // Use 'postCreateCommand' to run commands after the container is created.
  "postCreateCommand": "bash -i -c 'post_create_command' || true",

  // A command to run each time the container is successfully started.
  "postStartCommand": "bash -i -c 'post_start_command' || true",

  // Connect as root. To use non root user: https://aka.ms/dev-containers-non-root.
  "remoteUser": "root"
}
stuartleeks commented 1 year ago

@0xGosu it should only rebuild the image if the image is stale. If this is for a public repo, are you able to share a link to a workflow run that had a rebuild?

0xGosu commented 1 year ago

hi @stuartleeks unfortunately my org repos are private. I can share a portion of the workflow as below:

      - name: Build and run Dev Container task
        uses: devcontainers/ci@v0.3
        with:
          cacheFrom: org/pre-built-image:latest
          push: never
          runCmd: |
            cd ${{ inputs.CI_WORKING_DIRECTORY }}
            make .ci-test-main
          env: |
            CI_EVENT_NAME=${{ github.event_name }}
            SOURCE_BRANCH=${{ github.ref_name }}
            SOURCE_COMMIT=${{ github.sha }}
            SERVICE_NAME=${{ inputs.SERVICE_NAME }}
            AWS_REGION=${{ inputs.AWS_REGION }}
            AWS_ACCESS_KEY_ID=${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
            AWS_SECRET_ACCESS_KEY=${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
            GITHUB_TOKEN
        env: # Set an environment variable
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

As i mentioned, every time this github action step run, it always rebuilt the devcontainer image (which took around 1-2 minutes) with the command like this one (i copied it from the log) :

[2023-04-05T10:14:57.824Z] Start: Run: docker buildx build --load --build-arg _DEV_CONTAINERS_BASE_IMAGE=org/pre-built-image:latest --target dev_containers_target_stage -t vsc-project-service-3d3ad6e6f921a667ed221839e65f8b4161d988df06404b391eb3dc7555b3f83a-features -f /tmp/devcontainercli-runner/container-features/0.36.0-1680689697809/Dockerfile.extended /tmp/devcontainercli-runner/empty-folder

I just want to propose an option to completely disable this behavior as it is not required to rebuild the devcontainer image in my usecase

cjam commented 4 months ago

I have a similar issue. I was expecting that once the devcontainer and all it's features are built and pushed that we would be able to spin up the dev container quickly in subsequent actions within other repos. We have quite a few features in our devcontainer and our container takes around 4-5 mins to build every time we stand it up within our CI.

I can see in the logs that it is finding our existing image (within ghcr) but then it turns around and pulls in all of the features and installs them all again.

Even if there was a way to get everything cached, it would be slower than not even trying to build but would be much faster than what I have right now.

cjam commented 4 months ago

@0xGosu

I just wanted to share an update as I was finally able to get things figured out on my end and took my build from 12 minutes down to ~6.

I had only stumbled across the solution as I was working on another repo and then went through to find the differences. In my case, there were a few things that seemed to play a part in the cache invalidation. I'm not certain what exactly did it, but I can share the end result.

I'm using a docker-compose setup for my devcontainer with several services that get stood up.

version: '3.8'

services:
  # Dev Container
  dev:
    # The image that is being pushed for this dev-container
    image: ghcr.io/IMAGE_NAME:latest
    build:
      cache_from:
        - ghcr.io/IMAGE_NAME:latest
      context: .
      #sets Docker file unless .env file (in same folder has an override)
      dockerfile: ${DOCKERFILE:-Dockerfile} 
FROM mcr.microsoft.com/devcontainers/javascript-node:20

# I PREVIOUSLY HAD BEEN COPYING SOME FILES (NPMRC) IN HERE WHICH I THINK WAS INVALIDATING THE CACHE
# INSTEAD I MOVED THEM INTO THE DEVCONTAINER.JSON 
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/javascript-node-postgres
{
  "name": "Dev Container",
  "dockerComposeFile": "docker-compose.yml",
  "service": "dev",
  "workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}",
  "mounts": [
    // See: https://www.kenmuse.com/blog/dev-containers-and-node_modules/
    // ! the ${localEnv:VARIABLE:default value} syntax is used here to allow us to bind a folder volume within
    // ! github actions / CI environment so that we can cache the node modules to speed up build performance
    // ! by default (i.e. local development) it will bind to the node_modules volume defined in the volumes section of
    // ! the ./docker-compose.yml file
    "source=${localEnv:NODE_MODULE_MOUNT:node_modules,type=volume},target=${containerWorkspaceFolder}/node_modules",
    "source=${localEnv:YARN_CACHE_MOUNT:yarn_cache,type=volume},target=/home/node/.yarn",
    "source=.dapr/components,target=/home/node/.dapr/components",
    "source=.dapr/config.yaml,target=/home/node/.dapr/config.yaml",
    "source=./assets/.npmrc,target=/home/node/.npmrc",
  ],
  // These variables are made available to all processes spun up in the devcontainer
  "containerEnv": {
    "SHELL": "/bin/zsh",
    "YARN_CACHE_FOLDER": "/home/node/.yarn",
    "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",
    "NPM_TOKEN": "${localEnv:GITHUB_TOKEN}",
    "NODE_AUTH_TOKEN": "${localEnv:GITHUB_TOKEN}",
  },
  // This will pull environment variables from the local machine and make them available to vscode
  // and terminals etc
  "remoteEnv": {
    "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",
    "NPM_TOKEN": "${localEnv:GITHUB_TOKEN}",
    "NODE_AUTH_TOKEN": "${localEnv:GITHUB_TOKEN}"
  },
  "onCreateCommand": {
    "node_modules_perms": "sudo chown node node_modules",
    "yarn_cache_perms": "sudo chown node $YARN_CACHE_FOLDER",
    "npmrc_perms": "sudo chown node ~/.npmrc"
  },
  "features": {
    "ghcr.io/devcontainers/features/docker-in-docker:2": {
      "moby": true,
      "azureDnsAutoDetection": true,
      "installDockerBuildx": true,
      "version": "latest",
      "dockerDashComposeVersion": "v2"
    },
    "ghcr.io/devcontainers/features/terraform:1": {
      "installSentinel": true,
      "installTFsec": true,
      "installTerraformDocs": true,
      "version": "latest",
      "tflint": "0.46.1", // Latest version of tflint caused issues, see: https://github.com/devcontainers/features/issues/581
      "terragrunt": "0.55.7" // Latest version of terragrunt caused issue
    },
    "ghcr.io/dapr/cli/dapr-cli:0": {
      "version": "latest"
    },
    "ghcr.io/devcontainers/features/github-cli:1": {},
    "ghcr.io/devcontainers/features/java:1.2": {},
    "ghcr.io/devcontainers/features/azure-cli:1": {},
    "ghcr.io/robbert229/devcontainer-features/postgresql-client:1": {
      "version": "13" // Ideally, this should match the version of postgres that we are spinning up in the docker-compose file
    }
  },
  "customizations": {
    "vscode": {
      "extensions": [
        "ckolkman.vscode-postgres",
        "Angular.ng-template",
        "esbenp.prettier-vscode@9.14.0", // set to this version to avoid test runner breakage
        "firsttris.vscode-jest-runner",
        "dbaeumer.vscode-eslint",
        "nrwl.angular-console",
        "aaron-bond.better-comments",
        "mikestead.dotenv",
        "k--kato.docomment",
        "mrmlnc.vscode-scss",
        "eliostruyf.vscode-typescript-exportallmodules",
        "redhat.vscode-yaml",
        "mhutchie.git-graph",
        "eamodio.gitlens",
        "ms-azuretools.vscode-dapr",
        "oouo-diogo-perdigao.docthis",
        "bierner.markdown-mermaid",
        "yzhang.markdown-all-in-one",
        "donjayamanne.githistory",
        "joshbolduc.commitlint"
        // For .net in the future if we need
        // "bilal-arikan.csharp-auto-formatter",
        // "ms-dotnettools.csharp",
        // "kreativ-software.csharpextensions",
      ],
      "settings": {
        "terminal.integrated.defaultProfile.linux": "zsh",
        "extensions.autoUpdate": false, //set to false updates need to be explicit
        "extensions.autoCheckUpdates": false
      }
    }
  }
}

I think the key was removing the copy commands from my dockerfile and moving them into the devcontainer.json.

Also my CI looks like:

- name: Build
        uses: devcontainers/ci@v0.3
        env:
          # See devcontainer.json->mounts, essentially mounting a folder that we can cache into
          # the node modules
          NODE_MODULE_MOUNT: '${{ env.nodeModulesPath }}'
          # These environment variables are defined within the .devcontainer.json > remoteEnv
          # Which means that if they exist they will be bound do the dev container instance
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          imageName: ${{ env.registryRoot }}/aed/devcontainer
          push: never

          runCmd: |
            echo "running in container"

A lot to digest here, but just thought I'd share it in case that it helps.