Running Kaniko w/ options --no-push=true --cache=true and --cache-dir="..." leads to error

[stawr93]

I'd like to use caching to directory without publishing layers to any repo/registry.

Actual behavior Using --no-push=true --cache=true --cache-dir=/some-path options in combine leads to an error:

Error: cache flags invalid: if using cache with --no-push, specify cache repo with --cache-repo

Expected behavior In case of --no-push=true --cache=true --cache-dir=/some-path Kaniko will cache layers into provided directory and will not push them into any repo.

To Reproduce Steps to reproduce the behavior:

1. Run /kaniko/executor --context "<context>" --dockerfile "<path>/Dockerfile" --destination "<some custom tag>" --no-push --cache=true --cache-dir=<some path>

Additional Information

• Kaniko Image (fully qualified with digest): gcr.io/kaniko-project/executor:v1.9.0-debug

  Triage Notes for the Maintainers

  Description	Yes/No
  Please check if this a new feature you are proposing	- [ ]
  Please check if the build works in docker but not in kaniko	- [ ]
  Please check if this error is seen when you use --cache flag	- [x]
  Please check if your dockerfile is a multistage dockerfile	- [ ]

[tspearconquest]

Hi, I'm afraid that's not what --cache-dir= is for.

Flag --cache-dir Set this flag to specify a local directory cache for base images. Defaults to /cache.

However, this is further clarified in the section on caching base images: https://github.com/GoogleContainerTools/kaniko#caching-base-images

kaniko can cache images in a local directory that can be volume mounted into the kaniko pod. To do so, the cache must first be populated, as it is read-only.

(bolding is mine)

Since the --cache-dir= path is read-only, it should already have a base image in it for use by kaniko, and is intended to avoid downloading an image from dockerhub repeatedly.

If you're simply trying to avoid downloading base images, I recommend you review the link above. If you're trying to build images into this cache directory, you'll want to play around with the kaniko warmer image by using it to retrieve some random images from dockerhub and figure out how the cache looks when it is populated by that tool, then configure your kaniko command line to output the image the same way.

[aaron-prindle]

@stawr93 does the information @tspearconquest provided help clarify things? Is there still discussion needed or a possible kaniko bug here given this info? Currently I believe this is working as intended IIUC

[stawr93]

yeah, @tspearconquest 's answer been useful. It might have been my mistake. I think the issue can be closed/resolved.

In the end I decided to use the registry cache as it is the most simple way to share intermediate build artifacts between pipeline runs.

[Chococrok]

Hello, I think the issue is still valid. I used the warmer to populate the cache then used:

docker run --rm -v $PWD:/workspace -it gcr.io/kaniko-project/executor:v1.19.2-debug \
  --context "/workspace" \
  --dockerfile "/workspace/Dockerfile" \
  --tar-path /workspace/result.tar \
  --destination=test \
  --no-push \
  --cache=true \
  --cache-dir "/workspace/cache"

But kaniko is still complaining that I should use --cache-repo

[Chococrok]

This update fix the thing:

// cacheFlagsValid makes sure the flags passed in related to caching are valid
func cacheFlagsValid() error {
    if !opts.Cache {
        return nil
    }
    // If --cache=true and --no-push=true, then cache repo must be provided
    // since cache can't be inferred from destination
    if opts.CacheRepo == "" && opts.CacheDir == "" && opts.NoPush {
        return errors.New("if using cache with --no-push, specify cache repo with --cache-repo or cache dir with --cache-dir")
    }
    return nil
}

[reitzig]

FWIW: For creating reproducible MWEs for issues such as #1246, an "offline" way to combine --no-push with --cache would be very helpful. In my mind, tying the MWE to a repository makes it both less minimal and less reproducible.

Actually, I for one don't even know which --cache-repo to set that both me and maintainers would be able to use without potentially introducing further problems. 🤷