Warmer does not work with private images

[jamesatha]

Actual behavior I have a pod running the cache warmer. It has multiple --image arguments. The public images download just fine but the rest silently fail. I have made sure the docker config is mounted at /kaniko/.docker/config.json and the DOCKER_CONFIG environment variable is set to /kaniko/.docker/

Expected behavior Should use the credentials specified to download the private image.

To Reproduce Steps to reproduce the behavior:

1. Create a pod with for the cache warmer. Make sure there are some private images. In the list of images. Make sure that the docker config is properly mounted.
2. See that the private image was not downloaded

Additional Information Using gcr.io/kaniko-project/warmer:latest

[mamoit]

I'm using gitlab ci and in order to configure docker it actually requires a shell. Having a debug image for the warmer (as we have for kaniko itself) would give a lot of flexibility for this kind of use cases where some pre-configuring is needed.

[mamoit]

497 should allow to setup the credentials prior to warming up the cache.

[mamoit]

I have built a debug kaniko image with the warmer inside of it, and I can verify what @jamesatha said, I try to pull the images with the docker config setup (that works with kaniko), but it silently fails when trying to pull a image from a private repo.

[pdemagny]

I also confirm that even when using #497, it still does not work with gitlab ci private registry :( It would be awesome if it did !

[efenex]

same issue here, using debug image in combination with gitlab and warmer fails to pull private images, even though GOOGLE_APPLICATION_CREDENTIALS environment variable is configured and exported:

$ /kaniko/warmer --cache-dir=/cache --image=$CI_REGISTRY/private/base:1.4.3.1 --image=node:8-alpine
Failed warming cache: Failed to retrieve image: gitlab:4567/private/base:1.4.3.1: DENIED: access forbidden

[Enelar]

Confirmed. My workaround:

echo "FROM $SOURCE" > Dockerfile.try-pull
export IMAGE_NOT_EXISTS=$(executor --dockerfile Dockerfile.try-pull --no-push -v panic )
if [[ $IMAGE_NOT_EXISTS ]]
then
  # ...
fi

[Enelar]

@priyawadhwa please tag as issue/bug. This isn't documented behavior. Lost several hours debugging build scripts because it wasn't mentioned that warmer is not using docker login configs.

Our build pipeline have several layers of caching partial images, because building from scratch taking tens of minutes. After migrating from dind to kaniko, for some modules build process broke out.

[priyawadhwa]

Hey @Enelar I've tagged the issue. It looks like this is happening because we try to pull the image without credentials. We should be checking for credentials, as we do in the executor here.

If anyone is interested in submitting a PR, feel free to cc me and I can take a look at it.