aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.63k stars 3.91k forks source link

(core): Allow DockerImage.fromBuild to opt out of fingerprint logic #14734

Open TikiTDO opened 3 years ago

TikiTDO commented 3 years ago

Add an option in DockerImage.fromBuild to allow skipping the fingerprint generation step.

Use Case

Currently when this code builds a docker image it runs a function to ensure that an image has a stable hash. This function reads every file in docker source directory which can be excessively slow for big directories.

I only use the image as in another bundling step to process an asset, so I will never care about it having a stable hash. Since I need to build the image from the root the fingerprint function eats up 75% of the total execution time, even though I have absolutely no need for that hash, or that image after I have used it for the bundling step it's meant to do.

Proposed Solution

Add a skipFingerprint option to DockerBuildOptions. Use that to skip the fingerprint step if it's set.


This is a :rocket: Feature Request

jogold commented 3 years ago

I only use the image as in another bundling step to process an asset

Can you detail your use case? Is it something for cp() maybe?

TikiTDO commented 3 years ago

@jogold The cp() method is an instance method on the DockerImage class, so I would still need to get an instance of the image class I'm using before I could call it. Granted, I could use that method to implement my bundling step a using a different flow, but that wouldn't solve the 11 second delay I experience when I call DockerImage.fromBuild(GIT_ROOT, { file: "./docker/Dockerfile.app-dependeces", }).

github-actions[bot] commented 2 years ago

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

rob3c commented 2 years ago

If only it were an 11 second delay for some of my images. This is completely unusable for certain use cases, and the fix seems straight-forward and useful. What is the CDK team's argument against the feature?

TikiTDO commented 2 years ago

@rob3c If you want to speed it up you can make a copy of their file that looks something like this:

import * as child_process from "child_process"

import * as crypto from "crypto"
import * as path from "path"

import * as cdk from "aws-cdk-lib"

/**
 * This hack exists because cdk.DockerImage.fromBuild does a super slow fingerprint step. The fast
 * version performs the exact same steps as the original code, but without the fingerprint.
 *
 * If this issue ever gains any traction this hack can be removed: https://github.com/aws/aws-cdk/issues/14734
 */
export default class FasterDockerImage extends cdk.DockerImage {
  /**
   * Builds a Docker image
   *
   * @param path The path to the directory containing the Docker file
   * @param options Docker build options
   */
  public static fromFastBuild(
    buildPath: string,
    options: cdk.DockerBuildOptions = {},
  ): cdk.DockerImage {
    const buildArgs = options.buildArgs || {}

    if (options.file && path.isAbsolute(options.file)) {
      throw new Error(`"file" must be relative to the docker build directory. Got ${options.file}`)
    }

    // Image tag derived from path and build options
    const input = JSON.stringify({ buildPath, ...options })
    const tagHash = crypto.createHash("sha256").update(input).digest("hex")
    const tag = `cdk-${tagHash}`

    const dockerArgs: string[] = [
      "build",
      "-t",
      tag,
      ...(options.file ? ["-f", path.join(buildPath, options.file)] : []),
      ...flatten(Object.entries(buildArgs).map(([k, v]) => ["--build-arg", `${k}=${v}`])),
      buildPath,
    ]

    dockerExec(dockerArgs)

    return new cdk.DockerImage(tag)
  }
}

function flatten(x: string[][]): string[] {
  return Array.prototype.concat([], ...x)
}

function dockerExec(
  args: string[],
  options?: child_process.SpawnSyncOptions,
): child_process.SpawnSyncReturns<Buffer | string> {
  const prog = process.env.CDK_DOCKER ?? "docker"
  const proc = child_process.spawnSync(
    prog,
    args,
    options ?? {
      stdio: [
        // show Docker output
        "ignore", // ignore stdio
        process.stderr, // redirect stdout to stderr
        "inherit", // inherit stderr
      ],
    },
  )

  if (proc.error) {
    throw proc.error
  }

  if (proc.status !== 0) {
    if (proc.stdout || proc.stderr) {
      throw new Error(
        `[Status ${proc.status}] stdout: ${proc.stdout
          ?.toString()
          .trim()}\n\n\nstderr: ${proc.stderr?.toString().trim()}`,
      )
    }
    throw new Error(`${prog} exited with status ${proc.status}`)
  }

  return proc
}