GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.82k stars 1.44k forks source link

Long builds fail with "UNAUTHORIZED: \"authentication required\"" #245

Closed JensGutermuth closed 5 years ago

JensGutermuth commented 6 years ago

steps to reproduce

mkdir -p ./kaniko-issue

cat > ./kaniko-issue/Dockerfile <<EOF
FROM debian:stable-slim
RUN sleep 1
EOF
# This will work as expected
docker run -v `pwd`/kaniko-issue/:/workspace \
    gcr.io/kaniko-project/executor:latest \
    -c /workspace -f Dockerfile \
    -d registry.example.com/does/not:matter \
    --tarPath /workspace/tarball.tar

cat > ./kaniko-issue/Dockerfile <<EOF
FROM debian:stable-slim
RUN sleep 360
EOF
# This will fail with "UNAUTHORIZED: \"authentication required\""
docker run -v `pwd`/kaniko-issue/:/workspace \
    gcr.io/kaniko-project/executor:latest \
    -c /workspace -f Dockerfile \
    -d registry.example.com/does/not:matter \
    --tarPath /workspace/tarball.tar

# rm -r ./kaniko-issue

additional obervations

I also ran tcpdump on the network interface of the container. I saw a quite a bit of traffic at the start (I assume pulling the image) and a single, short TLS connection to index.docker.io after sleep was done.

The issue seems to be gone (or at least takes substantially longer to arise) if I substitute debian:stable-slim with any image from my harbor (private docker registry) instance.

working theory

My working theory of the underlying cause based on those two observations is, that kaniko tries to fetch the image config of the base image using an expired bearer token. This config would normally be extended and included in the tarball or pushed to the registry.

priyawadhwa commented 6 years ago

Hey @Delphinator , I tried to repro this issue but was able to build the image as expected:

$ cat Dockerfile 
FROM debian:stable-slim
RUN sleep 360
$ ./run_in_docker.sh Dockerfile $(pwd) gcr.io/priya-wadhwa/test:test
time="2018-07-25T21:50:49Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/25 21:50:49 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-25T21:50:49Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/perf_event /sys/fs/cgroup/cpuset /sys/fs/cgroup/blkio /sys/fs/cgroup/freezer /sys/fs/cgroup/pids /sys/fs/cgroup/devices /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /root/.config/gcloud /proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-25T21:50:50Z" level=info msg="Unpacking layer: 0"
time="2018-07-25T21:50:50Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-25T21:50:50Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-25T21:50:50Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-25T21:50:50Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-25T21:50:50Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-25T21:50:51Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-25T21:50:51Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-25T21:50:53Z" level=info msg="cmd: /bin/sh"
time="2018-07-25T21:50:53Z" level=info msg="args: [-c sleep 360]"
time="2018-07-25T21:56:53Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-25T21:56:54Z" level=info msg="No files were changed, appending empty layer to config."
2018/07/25 21:56:55 mounted blob: sha256:7d1463d31d7e5ad679ea175cd72afede858628ca49598925a17410e452d5ccec
2018/07/25 21:56:55 mounted blob: sha256:b5186294b6f665381d964ff1e51910d9c03599009ca8a8a54a66607f44daf490
gcr.io/priya-wadhwa/test:test: digest: sha256:d1244e42b3de8475275900102535ca00efdc05c9bbb4882c78ff816215028ea6 size: 429

We fetch the image config of the base image before starting to run commands here, and it's downloaded along with the base image, so an expired token shouldn't be the issue.

When pushing the image after running commands, we also get a new auth token.

I'm not sure why you're hitting this error since I couldn't repro the issue, but the only thing I can think of right now is that it's an auth issue with the registry you're trying to push to?

JensGutermuth commented 6 years ago

That's odd. I just tried it again and I'm consistently getting errors when sleeping 6 minutes. I do remember 6 minutes being somewhat close to the threshold though. Maybe 10 minutes will do the job in your environment?

I'm not convinced it's an auth issue with the private registry.

Just to be sure, this is the exact image I used and the docker version:

# docker inspect gcr.io/kaniko-project/executor:latest
[
    {
        "Id": "sha256:20b09335e9e6883fe370dff009b33a09530c1727835a96042f6733c506fba3ca",
        "RepoTags": [
            "gcr.io/kaniko-project/executor:latest"
        ],
        "RepoDigests": [
            "gcr.io/kaniko-project/executor@sha256:501056bf52f3a96f151ccbeb028715330d5d5aa6647e7572ce6c6c55f91ab374"
        ]
# [snip]

# docker --version
Docker version 18.06.0-ce, build 0ffa825

And logs of my most recent attempts:

trying to push to a registry with valid credentials in /root/.docker/config.json

sleep 1:

+ docker run -v /tmp/kaniko-issue/:/workspace -v /root/.docker:/root/.docker:ro gcr.io/kaniko-project/executor:latest -c /workspace -f Dockerfile -d [REDACTED]/[REDACTED]/test:latest
time="2018-07-26T09:05:44Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/26 09:05:44 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-26T09:05:45Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/perf_event /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/pids /sys/fs/cgroup/devices /sys/fs/cgroup/cpuset /dev/mqueue /workspace /root/.docker /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-26T09:05:45Z" level=info msg="Unpacking layer: 0"
time="2018-07-26T09:05:46Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-26T09:05:46Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-26T09:05:46Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-26T09:05:46Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-26T09:05:47Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-26T09:05:49Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-26T09:05:49Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T09:05:50Z" level=info msg="cmd: /bin/sh"
time="2018-07-26T09:05:50Z" level=info msg="args: [-c sleep 1]"
time="2018-07-26T09:05:51Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T09:05:52Z" level=info msg="No files were changed, appending empty layer to config."
2018/07/26 09:05:52 pushed blob sha256:b5186294b6f665381d964ff1e51910d9c03599009ca8a8a54a66607f44daf490
2018/07/26 09:05:54 pushed blob sha256:7d1463d31d7e5ad679ea175cd72afede858628ca49598925a17410e452d5ccec
docker.[REDACTED]/[REDACTED]/test:latest: digest: sha256:d1244e42b3de8475275900102535ca00efdc05c9bbb4882c78ff816215028ea6 size: 429

sleep 360:

+ docker run -v /tmp/kaniko-issue/:/workspace -v /root/.docker:/root/.docker:ro gcr.io/kaniko-project/executor:latest -c /workspace -f Dockerfile -d [REDACTED]/[REDACTED]/test:latest
time="2018-07-26T08:53:02Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/26 08:53:02 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-26T08:53:03Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/perf_event /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/pids /sys/fs/cgroup/devices /sys/fs/cgroup/cpuset /dev/mqueue /workspace /root/.docker /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-26T08:53:03Z" level=info msg="Unpacking layer: 0"
time="2018-07-26T08:53:04Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-26T08:53:04Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-26T08:53:04Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-26T08:53:04Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-26T08:53:04Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-26T08:53:06Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-26T08:53:06Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T08:53:08Z" level=info msg="cmd: /bin/sh"
time="2018-07-26T08:53:08Z" level=info msg="args: [-c sleep 360]"
time="2018-07-26T08:59:08Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T08:59:09Z" level=info msg="No files were changed, appending empty layer to config."
2018/07/26 08:59:09 pushed blob sha256:b5186294b6f665381d964ff1e51910d9c03599009ca8a8a54a66607f44daf490
time="2018-07-26T08:59:09Z" level=error msg="UNAUTHORIZED: \"authentication required\""

just building a tarball

sleep 1:

+ docker run -v /tmp/kaniko-issue/:/workspace gcr.io/kaniko-project/executor:latest -c /workspace -f Dockerfile -d registry.example.com/does/not:matter --tarPath /workspace/tarball.tar
time="2018-07-26T09:05:56Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/26 09:05:56 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-26T09:05:57Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/perf_event /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/pids /sys/fs/cgroup/devices /sys/fs/cgroup/cpuset /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-26T09:05:57Z" level=info msg="Unpacking layer: 0"
time="2018-07-26T09:05:57Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-26T09:05:57Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-26T09:05:57Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-26T09:05:58Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-26T09:05:58Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-26T09:05:59Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-26T09:05:59Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T09:06:01Z" level=info msg="cmd: /bin/sh"
time="2018-07-26T09:06:01Z" level=info msg="args: [-c sleep 1]"
time="2018-07-26T09:06:02Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T09:06:03Z" level=info msg="No files were changed, appending empty layer to config."

sleep 360:

+ docker run -v /tmp/kaniko-issue/:/workspace gcr.io/kaniko-project/executor:latest -c /workspace -f Dockerfile -d registry.example.com/does/not:matter --tarPath /workspace/tarball.tar
time="2018-07-26T08:59:11Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/26 08:59:11 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-26T08:59:12Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/perf_event /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/blkio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/pids /sys/fs/cgroup/devices /sys/fs/cgroup/cpuset /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-26T08:59:13Z" level=info msg="Unpacking layer: 0"
time="2018-07-26T08:59:14Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-26T08:59:14Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-26T08:59:14Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-26T08:59:14Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-26T08:59:14Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-26T08:59:16Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-26T08:59:16Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T08:59:17Z" level=info msg="cmd: /bin/sh"
time="2018-07-26T08:59:17Z" level=info msg="args: [-c sleep 360]"
time="2018-07-26T09:05:17Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-26T09:05:19Z" level=info msg="No files were changed, appending empty layer to config."
time="2018-07-26T09:05:19Z" level=error msg="UNAUTHORIZED: \"authentication required\""

This also fails if I mount the docker configuration as above and change the target accordingly.

priyawadhwa commented 6 years ago

I tried again with 10 minutes, and it still worked fine:

time="2018-07-27T19:54:37Z" level=info msg="Unpacking filesystem of debian:stable-slim..."
2018/07/27 19:54:37 No matching credentials found for index.docker.io, falling back on anonymous
time="2018-07-27T19:54:38Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/perf_event /sys/fs/cgroup/cpuset /sys/fs/cgroup/blkio /sys/fs/cgroup/freezer /sys/fs/cgroup/pids /sys/fs/cgroup/devices /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /root/.config/gcloud /proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-07-27T19:54:38Z" level=info msg="Unpacking layer: 0"
time="2018-07-27T19:54:39Z" level=info msg="Not adding /dev because it is whitelisted"
time="2018-07-27T19:54:39Z" level=info msg="Not adding /etc/hostname because it is whitelisted"
time="2018-07-27T19:54:39Z" level=info msg="Not adding /etc/resolv.conf because it is whitelisted"
time="2018-07-27T19:54:39Z" level=info msg="Not adding /proc because it is whitelisted"
time="2018-07-27T19:54:39Z" level=info msg="Not adding /sys because it is whitelisted"
time="2018-07-27T19:54:40Z" level=info msg="Not adding /var/run because it is whitelisted"
time="2018-07-27T19:54:40Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-27T19:54:41Z" level=info msg="cmd: /bin/sh"
time="2018-07-27T19:54:41Z" level=info msg="args: [-c sleep 600]"
time="2018-07-27T20:04:41Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-07-27T20:04:43Z" level=info msg="No files were changed, appending empty layer to config."
2018/07/27 20:04:44 mounted blob: sha256:7d1463d31d7e5ad679ea175cd72afede858628ca49598925a17410e452d5ccec
2018/07/27 20:04:44 mounted blob: sha256:b5186294b6f665381d964ff1e51910d9c03599009ca8a8a54a66607f44daf490
gcr.io/priya-wadhwa/test:test: digest: sha256:d1244e42b3de8475275900102535ca00efdc05c9bbb4882c78ff816215028ea6 size: 429

If you could submit a PR with a Dockerfile that breaks our CI we might be able to explore this issue a bit more. You'd just have to add another Dockerfile in the dockerfiles directory

dlorenc commented 6 years ago

Ping @Delphinator any luck getting a repro?

MnrGreg commented 6 years ago

I'm experiencing the same error message after 6mins and 20seconds of a NPM container build.

INFO[0281] Adding whiteout for /app/node_modules/extract-zip INFO[0281] Adding whiteout for /app/node_modules/es6-promise/auto.js INFO[0320] Storing source image from stage 0 at path /kaniko/stages/0 ERRO[0364] UNAUTHORIZED: "authentication required"

NPM Docker file:

# Stage 0, "build-stage", based on Node.js, to build and compile the frontend
FROM tiangolo/node-frontend:10 as build-stage
WORKDIR /app
COPY package*.json /app/
RUN npm install
COPY ./ /app/
ARG configuration=production
RUN npm run build -- --output-path=./dist/out --configuration $configuration

# Stage 1, based on Nginx, to have only the compiled app, ready for production with Nginx
FROM nginx:1.15
COPY --from=build-stage /app/dist/out/ /usr/share/nginx/html
# Copy the default nginx.conf provided by tiangolo/node-frontend
COPY --from=build-stage /nginx.conf /etc/nginx/conf.d/default.conf

I can however build and push the following sleepy container without issue.

FROM nginx:1.15
RUN sleep 360

INFO[0009] Taking snapshot of full filesystem... INFO[0010] Executing 0 build triggers INFO[0010] cmd: /bin/sh INFO[0010] args: [-c sleep 360] INFO[0370] Taking snapshot of full filesystem... INFO[0371] No files were changed, appending empty layer to config. 2018/08/02 20:30:52 mounted blob: sha256:32d9726baeef1d8d626fa618fdc76200248b83e6453d642c25bddee92326d5ab 2018/08/02 20:30:52 mounted blob: sha256:be8881be8156e4068e611fe956aba2b9593ebd953be14fb7feea6d0659aa3abe 2018/08/02 20:30:52 mounted blob: sha256:87e5e6f71297704d2e6724248ededfb71e994dcf2fd9568b309b2d294e6ee455 2018/08/02 20:30:55 pushed blob sha256:f823acf1d9c5e1ad81a45e1220dd5718bdeae8354f8d1fee0f2be5eb260d7d2b 2018/08/02 20:30:57 container-registry.XXX/stores/XXweb:timerv1: digest: sha256:7ac9d1c278db35acd78a1da0dc6cbe2486b5838bab7c6334ec3879786dffe591 size: 754

Destination registry is Docker Trusted Registry 2.5.3

JensGutermuth commented 6 years ago

I'm experiencing the same error message after 6mins and 20seconds of a NPM container build.

I just pushed another attempt to reproduce in #267 with a two stage build. Hopefully it breaks.

I can however build and push the following sleepy container without issue.

I'm not 100% sure if 6 minutes are enough on a fast system. I did my experiments on a somewhat busy system with spinning disks. That does slow container startup and unpacking / packing down significantly. Maybe try 10 minutes just to be extra sure?

JensGutermuth commented 6 years ago

Another thought: I first encountered the issue while building images using the CI component of my private gitlab instance. I just created a new user there, added the SSH keys of @priyawadhwa and @dlorenc (github makes SSH keys public for some reason) to that user and pushed a repo, which reproduces the issue in gitlab CI. The repo and build logs are publicly accessible at https://gitlab.jensgutermuth.de/kaniko-issue-245/reproduce. Feel free to experiment (probably best on a branch, so you don't step on each others work).

The CI jobs in that repo first build kaniko using itself to facilitate adding debugging code or testing changes. Just change the git clone command in Dockerfile-kaniko. Using kaniko to build itself unfortunately requires a bit of hacky shell scripting and strategic moving around of files, since kaniko-build images cannot have files in /kaniko. Let's hope this won't cause issues.

JensGutermuth commented 6 years ago

Well, I'm completely confused now. Two stage builds seem to work for me (ignore the -o pipefail-stuff, that's just the gitlab runner expecting things).

MnrGreg commented 6 years ago

I did a bit more troubleshooting and noticed that when adding the 'COPY --from' the build succeeds 1/5 times with a 185 sleep, while without the 'COPY --from' it succeeds every time 5/5.

Decreasing the sleep time to below 60 seconds with the 'COPY --from' succeeds 5/5 times.

FROM container-registry.XX/stores/node-frontend:10 as build-stage
RUN sleep 185
FROM container-registry.XX/stores/nginx:1.15
COPY --from=build-stage /nginx.conf /etc/nginx/conf.d/default.conf
RUN echo "second stage is working"

INFO[0368] Not adding /kaniko/0/var/log/fontconfig.log because it was added by a prior layer INFO[0368] Unpacking layer: 2 ERRO[0368] UNAUTHORIZED: "authentication required"

@Delphinator could we include the 'COPY --from prior build' into you tests- seems it make play a part?

MnrGreg commented 6 years ago

Just got an UNAUTHORIZED with a 60 sleep and at a different unpacking stage. Seems to be quite inconsistent.

Also seem that this is during a Pull from the registry and not the final image Push.

INFO[0046] Taking snapshot of full filesystem... INFO[0053] Executing 0 build triggers INFO[0053] cmd: /bin/sh INFO[0053] args: [-c sleep 60] INFO[0113] Taking snapshot of full filesystem... INFO[0119] No files were changed, appending empty layer to config. INFO[0119] Storing source image from stage 0 at path /kaniko/stages/0 INFO[0282] trying to extract to /kaniko/0 INFO[0282] Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/cpuset /sys/fs/cgroup/cpu /sys/fs/cgroup/cpuacct /sys/fs/cgroup/blkio /sys/fs/cgroup/memory /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls /sys/fs/cgroup/perf_event /sys/fs/cgroup/net_prio /sys/fs/cgroup/hugetlb /sys/fs/cgroup/pids /sys/fs/cgroup/systemd /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /root/.docker/config.json /kaniko/ssl/certs/ca-certificates.crt/dev/console /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/cpuset /sys/fs/cgroup/cpu /sys/fs/cgroup/cpuacct /sys/fs/cgroup/blkio /sys/fs/cgroup/memory /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls /sys/fs/cgroup/perf_event /sys/fs/cgroup/net_prio /sys/fs/cgroup/hugetlb /sys/fs/cgroup/pids /sys/fs/cgroup/systemd /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /root/.docker/config.json /kaniko/ssl/certs/ca-certificates.crt /dev/console /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware] INFO[0282] Unpacking layer: 12 INFO[0283] Unpacking layer: 11 INFO[0316] Unpacking layer: 10 ERRO[0316] UNAUTHORIZED: "authentication required"

second run with timer at 10 seconds:

INFO[0329] Not adding /kaniko/0/app because it was added by a prior layer INFO[0329] Unpacking layer: 9 INFO[0352] Whiting out /kaniko/0/usr/bin/.wh.curl INFO[0355] Whiting out /kaniko/0/usr/share/doc/.wh.curl INFO[0373] Whiting out /kaniko/0/usr/share/man/man1/.wh.curl.1.gz INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.deb.debian.org_debian_dists_jessie-updates_InRelease INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.deb.debian.org_debian_dists_jessie-updates_main_binary-amd64_Packages.gz INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.deb.debian.org_debian_dists_jessie_Release INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.deb.debian.org_debian_dists_jessie_Release.gpg INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.deb.debian.org_debian_dists_jessie_main_binary-amd64_Packages.gz INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.lock INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/partial/.wh..wh..opq INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.security.debian.org_debian-security_dists_jessie_updates_InRelease INFO[0373] Whiting out /kaniko/0/var/lib/apt/lists/.wh.security.debian.org_debian-security_dists_jessie_updates_main_binary-amd64_Packages.gz INFO[0373] Whiting out /kaniko/0/var/lib/dpkg/info/.wh.curl.list INFO[0373] Whiting out /kaniko/0/var/lib/dpkg/info/.wh.curl.md5sums INFO[0373] Unpacking layer: 8 ERRO[0373] UNAUTHORIZED: "authentication required"

JensGutermuth commented 6 years ago

@Delphinator could we include the 'COPY --from prior build' into you tests- seems it make play a part?

Sure! Let's see what happens: https://gitlab.jensgutermuth.de/kaniko-issue-245/reproduce/pipelines/2056

JensGutermuth commented 6 years ago

Jop. A build w/o copy --from succeeds, while a build using copy --from fails.

MnrGreg commented 6 years ago

Lucky us! You hit the error on the first go. It seems to be somewhat sporadic for me.

JensGutermuth commented 6 years ago

It's been very consistent for me so far: 1st run 2nd run 3rd run

sap-svetter commented 6 years ago

Hi all,

I got the same issues after setting up some python software module (compiled with internet sources) on a base image. docker login succeeds without issues but kaniko still refers to "unauthorized".

Looking forward to a fix 👍

If I can do anything to help, tell me and I will do my very best to help you!

akhmerov commented 6 years ago

Could the duration of the push itself be relevant? The difference in upload speed could explain the lack of reproducibility.

BryanHunt commented 6 years ago

I have just run into this problem when running as part of a Jenkins build on Kubernetes. When I run locally (on my MacBook Pro) I'm able to build the Docker image just fine, but when I run as part of a Jenkins build on Kubernetes, the Kaniko container consumes a huge amount of memory (50+ GB) and after 14 minutes, it fails with:

error building image: UNAUTHORIZED: "authentication required"

At first, I thought it was a problem with trying to push the image to our local repository, so I added the --no-push option and I still get this error. Here's my build pipeline:

properties ([
      buildDiscarder(logRotator(numToKeepStr: '3')),
      disableConcurrentBuilds()
    ])

def label = "nv-ad-docs-${UUID.randomUUID().toString()}"

podTemplate(label: label, imagePullSecrets: ['docker'], yaml: """
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: nodejs
    image: <our-image-repo>/gpuwa-node:10-alpine-9
    imagePullPolicy: Always
    command:
    - cat
    tty: true
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true
    volumeMounts:
      - name: jenkins-docker-cfg
        mountPath: /root
    resources:
      {requests: {cpu: 1000m, memory: 100Gi}, limits: {cpu: 2000m, memory: 100Gi}}
  volumes:
  - name: jenkins-docker-cfg
    projected:
      sources:
      - secret:
          name: docker
          items:
            - key: .dockerconfigjson
              path: .docker/config.json

""")
{
  node(label) {
    stage('Build') {
      container('nodejs') {
        checkout scm
        sh 'npm install --only-production'
      }
    }

    stage("Assemble") {
      container(name: 'kaniko', shell: '/busybox/sh') {
        withEnv(['PATH+EXTRA=/busybox']) {
          sh '''#!/busybox/sh
          /kaniko/executor --dockerfile=`pwd`/Dockerfile --context=`pwd` --destination=<our-image-repo>/test-node:k8s --no-push
          '''
        }
      }
    }
  }
}

and here is my Dockerfile:

FROM node:10-alpine

WORKDIR /app
COPY . /app

EXPOSE 8080
CMD node index.js
brandon-bethke-timu commented 6 years ago

We are also experiencing this issue and it's a major roadblock. We do not know how to reproduce it consistently but it happens often. We are using kaniko in a gitlab ci pipeline.

Error: error building image: Error setting layer times: Error getting layer: UNAUTHORIZED: "authentication required"

We are using the debug image: gcr.io/kaniko-project/executor:debug and this is how we launch kaniko: /kaniko/executor --reproducible --dockerfile Dockerfile --context /mycontext --destination gcr.io/myplatform/myimage

BryanHunt commented 6 years ago

@brandon-bethke-neudesic I worked around the problem by switching to https://github.com/containers/buildah

nicolaferraro commented 6 years ago

I suspect the issue I'm experiencing in #372 is related. There are no "sleep" commands, just a build that is taking 5~10 mins.

tzununbekov commented 6 years ago

I run into this problem and figured out that "unauthorized" response is caused by expired anonymous token when build stage last longer than 5+ minutes, similar to docker/distribution#1589 though fix described there didn't help for my case

Issue may be reproduced with following Dockerfile:

FROM golang:1.10.1
WORKDIR /go/src/github.com/tzununbekov/simple-app/
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -v -o app
# Without following line build passes successfully 
RUN sleep 360

FROM scratch
COPY --from=0 /go/src/github.com/tzununbekov/simple-app .
ENTRYPOINT ["/app"]

and main.go next to the dockerfile with any simple code:

package main

import "fmt"

func main() {
    fmt.Println("Hello world!")
}

Managed to work this around by retrieving stage source image before getting its tarball in pkg/executor/build.go. I may create PR for this but token expiry handler would be a better solution here.

ianberinger commented 6 years ago

I'm impacted by this as well.

Thanks @tzununbekov for the pointer. I've debugged this further, this seems to be caused by https://github.com/google/go-containerregistry not refreshing expired Bearer tokens. Bearer tokens expire after a fixed duration (for docker.io after 300s). If a stage takes longer than that, saving the stage (which involves getting the compressed source image layers) will fail.

I've created a PR that fixes this: https://github.com/google/go-containerregistry/pull/283.

pieterlange commented 6 years ago

@tstromberg can you assign this issue to someone else so this fix gets shipped without undue delay? This makes kaniko unusable for building containers that take >5m to build.

dlorenc commented 6 years ago

@pieterlange it looks like the fix to the underlying issue is being reviewed now in go-containeregistry.

Once that's merged we can get the dependency updated here fairly easily.

priyawadhwa commented 6 years ago

Update: #388 was just merged, but I'll keep this issue open for a few days in case anyone continues to see this error.

Thanks @ianberinger for the fix!!

akhmerov commented 6 years ago

Is the fix included in the latest kaniko-project/executor?

priyawadhwa commented 6 years ago

yup, it should be in gcr.io/kaniko-project/executor:latest

yurrriq commented 6 years ago

388 fixed my long builds.

Edit: It looks like I spoke too soon...

error building image: getting stage builder for stage 1: no token in bearer response:
{"errors":[{"code":"DENIED","message":"access forbidden"}],"http_status":403}

Update: Extending the authorization token duration, per @pieterlange's suggestion in GitLab's container registry settings fixed it. Thanks, everyone!

pieterlange commented 6 years ago

I can confirm this fixed it for me!

@yurrriq make sure the token doesn't expire serverside either - gitlab-ci's expiry also happens to be 5 minutes, but you can easily up that in the admin console.

slavoutich commented 6 years ago

Works for me, thanks everyone!

kennethaasan commented 6 years ago

I can confirm the same. Good job guys!

AndreasBieber commented 5 years ago

Hi,

Unfortunately we still have this problem. We use Artifactory as a Docker Registry. When caching layers we still get a 401. Sometimes when pushing the 3rd or 4th layer. Sometimes, however, only when pushing the final layer. Also a deactivation of the cache does not help.

This problem occurs only on long running builds.

error building image: error building stage: failed to push to destination private-registry/service/cache:7ecab1d646fdcf96a55cdfb66938bb1f5c565e510729a480a26b992b309e6069: no token in bearer response:
{
  "errors" : [ {
    "status" : 401,
    "message" : "Authentication is required"
  } ]
}
yurrriq commented 5 years ago

@AndreasBieber, perhaps you need to replace /root/ with /kaniko/, as described on this potentially relevant GitLab issue.

AndreasBieber commented 5 years ago

@yurrriq /kaniko is already in place, but do not work.

AndreasBieber commented 5 years ago

Forgotten to mention: This problem only occurs with multistage builds.

bobcatfish commented 5 years ago

Sorry to hear this is still happening @AndreasBieber :(

Is there any chance you could provide us with a instructions we could use to reproduced the issue you are still seeing, e.g. maybe similar to the example in this issue's description?

AndreasBieber commented 5 years ago

@bobcatfish: Sorry, it was my bad. We are building our own image of kaniko for the GitLab CI Runner with custom scripts.

...
FROM gcr.io/kaniko-project/executor:v0.6.0 as kaniko
...
FROM base
COPY --from=kaniko /kaniko/executor /kaniko/

Since we also build this custom image with kaniko, the docker auth config was accidentally overwritten during the build process.

Now it works like charm. Thanks anyway.

priyawadhwa commented 5 years ago

Awesome, I'm going to go ahead and close this issue since it seems like #388 fixed it. If anyone experiences this again please comment on this thread or open another issue!

akhmerov commented 5 years ago

I seem to observe a regression between debug-v0.7.0 and debug-v0.8.0, debug-v0.8.0 fails with the same error message (authentication is required).

slavoutich commented 5 years ago

@akhmerov Replace /root/.docker -> /kaniko/.docker in .gitlab-ci.yml, this issue has nothing to do with your issue.

Caligatio commented 5 years ago

I hate to re-open old bugs but recently just encountered this with a private registry. Quick builds (< 5 minutes) are uploaded just fine but longer builds (> 5 minutes) hit a 401 on upload. Unfortunately I don't run the registry so I cannot confirm the bearer token lifetime.

Reading the comments above, I'm confused whether this bug was actually fixed or people just extended the server-side token lifetimes.

JensGutermuth commented 5 years ago

I'm fairly certain this was actually fixed. The original issue was reproducible with the central registry at registry.docker.io and I don't think they changed their token lifetimes. https://github.com/GoogleContainerTools/kaniko/issues/245#issuecomment-427678387 explains the fix and links to the relevant PR.

culdev commented 5 years ago

@Caligatio I kept getting this error in multi stage docker builds when running in GitLab CI. Solved it by moving the .docker folder like slavoutich suggested, e.g:

Build:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  variables:
    DOCKER_CONFIG: /kaniko/.docker
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:latest --destination $CI_REGISTRY_IMAGE/builds:$(date +'%Y-%m-%d')
Caligatio commented 5 years ago

I'm currently hitting this with all my builds that take longer than 5-ish minutes. I am confused by yurrriq's response as it seems like the fix didn't work for them so they just extended the token lifetime.

@culdev: I can confirm I have the /kaniko/.docker/config.json file in place and it works for quick builds.

EDIT: If it somehow matters, I'm also sitting behind a proxy.

EDIT2: Turns out my issue was caused by a misbehaving Registry auth service. The symptoms looked similar but was a completely different problem. Sorry all!

Ghostwritten commented 2 years ago

my yaml:

cat kaniko-dir2.yaml 
---

apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
#    image: gcr.io/kaniko-project/executor:debug
    image: registry.aliyuncs.com/kaniko-project/executor:latest
    args: ["--dockerfile=/workspace/Dockerfile",
            "--context=dir://workspace",
            "--destination=docker.io/zongxun/devops-toolkit:1.0.0"]
    volumeMounts:
      - name: docker-config
        mountPath: /kaniko/.docker
      - name: workspace
        mountPath: /workspace
  restartPolicy: Never
  volumes:
  - name: docker-config
    configMap:
      name: docker-config
  - name: workspace
    hostPath: 
      path: /root/kaniko/kaniko-demo 

yes, it is /kaniko/.docker not /root/.docker, but still error !

$ k logs kaniko -f
INFO[0000] Downloading base image klakegg/hugo:0.78.2-alpine 
INFO[0007] Extracting layer 0                           
INFO[0013] Extracting layer 1                           
INFO[0048] Extracting layer 2                           
INFO[0061] Taking snapshot of full filesystem...        
INFO[0066] RUN apk add -U git                           
INFO[0066] cmd: /bin/sh                                 
INFO[0066] args: [-c apk add -U git]                    
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/7) Installing ca-certificates (20191127-r4)
(2/7) Installing nghttp2-libs (1.41.0-r0)
(3/7) Installing libcurl (7.79.1-r0)
(4/7) Installing expat (2.2.9-r1)
(5/7) Installing pcre2 (10.35-r0)
(6/7) Installing git (2.26.3-r0)
(7/7) Installing git-bash-completion (2.26.3-r0)
Executing busybox-1.31.1-r19.trigger
Executing ca-certificates-20191127-r4.trigger
OK: 30 MiB in 30 packages
INFO[0077] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0077] COPY . /src                                  
INFO[0077] Creating directory /src                      
INFO[0077] Copying file workspace/.dockerignore to /src/.dockerignore 
INFO[0077] Creating directory /src/.git                 
INFO[0077] Copying file workspace/.git/HEAD to /src/.git/HEAD 
INFO[0077] Creating directory /src/.git/branches        
INFO[0077] Copying file workspace/.git/config to /src/.git/config 
INFO[0077] Copying file workspace/.git/description to /src/.git/description 
INFO[0077] Creating directory /src/.git/hooks           
INFO[0077] Copying file workspace/.git/hooks/applypatch-msg.sample to /src/.git/hooks/applypatch-msg.sample 
INFO[0077] Copying file workspace/.git/hooks/commit-msg.sample to /src/.git/hooks/commit-msg.sample 
INFO[0077] Copying file workspace/.git/hooks/fsmonitor-watchman.sample to /src/.git/hooks/fsmonitor-watchman.sample 
INFO[0077] Copying file workspace/.git/hooks/post-update.sample to /src/.git/hooks/post-update.sample 
INFO[0077] Copying file workspace/.git/hooks/pre-applypatch.sample to /src/.git/hooks/pre-applypatch.sample 
INFO[0077] Copying file workspace/.git/hooks/pre-commit.sample to /src/.git/hooks/pre-commit.sample 
INFO[0077] Copying file workspace/.git/hooks/pre-push.sample to /src/.git/hooks/pre-push.sample 
INFO[0077] Copying file workspace/.git/hooks/pre-rebase.sample to /src/.git/hooks/pre-rebase.sample 
INFO[0077] Copying file workspace/.git/hooks/pre-receive.sample to /src/.git/hooks/pre-receive.sample 
INFO[0077] Copying file workspace/.git/hooks/prepare-commit-msg.sample to /src/.git/hooks/prepare-commit-msg.sample 
INFO[0077] Copying file workspace/.git/hooks/update.sample to /src/.git/hooks/update.sample 
INFO[0077] Copying file workspace/.git/index to /src/.git/index 
INFO[0077] Creating directory /src/.git/info            
INFO[0077] Copying file workspace/.git/info/exclude to /src/.git/info/exclude 
INFO[0077] Creating directory /src/.git/logs            
INFO[0077] Copying file workspace/.git/logs/HEAD to /src/.git/logs/HEAD 
INFO[0077] Creating directory /src/.git/logs/refs       
INFO[0077] Creating directory /src/.git/logs/refs/heads 
INFO[0077] Copying file workspace/.git/logs/refs/heads/master to /src/.git/logs/refs/heads/master 
INFO[0077] Creating directory /src/.git/logs/refs/remotes 
INFO[0077] Creating directory /src/.git/logs/refs/remotes/origin 
INFO[0077] Copying file workspace/.git/logs/refs/remotes/origin/HEAD to /src/.git/logs/refs/remotes/origin/HEAD 
INFO[0077] Creating directory /src/.git/objects         
INFO[0077] Creating directory /src/.git/objects/info    
INFO[0077] Creating directory /src/.git/objects/pack    
INFO[0077] Copying file workspace/.git/objects/pack/pack-1f15da2c86849649fce7365eb92fb1fcfcc54848.idx to /src/.git/objects/pack/pack-1f15da2c86849649fce7365eb92fb1fcfcc54848.idx 
INFO[0077] Copying file workspace/.git/objects/pack/pack-1f15da2c86849649fce7365eb92fb1fcfcc54848.pack to /src/.git/objects/pack/pack-1f15da2c86849649fce7365eb92fb1fcfcc54848.pack 
INFO[0077] Copying file workspace/.git/packed-refs to /src/.git/packed-refs 
INFO[0077] Creating directory /src/.git/refs            
INFO[0077] Creating directory /src/.git/refs/heads      
INFO[0077] Copying file workspace/.git/refs/heads/master to /src/.git/refs/heads/master 
INFO[0077] Creating directory /src/.git/refs/remotes    
INFO[0077] Creating directory /src/.git/refs/remotes/origin 
INFO[0077] Copying file workspace/.git/refs/remotes/origin/HEAD to /src/.git/refs/remotes/origin/HEAD 
INFO[0077] Creating directory /src/.git/refs/tags       
INFO[0077] Copying file workspace/.gitignore to /src/.gitignore 
INFO[0077] Copying file workspace/.gitmodules to /src/.gitmodules 
INFO[0077] Copying file workspace/.gitpod.Dockerfile to /src/.gitpod.Dockerfile 
INFO[0077] Copying file workspace/.gitpod.yml to /src/.gitpod.yml 
INFO[0077] Copying file workspace/.helmignore to /src/.helmignore 
INFO[0077] Copying file workspace/APKINDEX.tar.gz to /src/APKINDEX.tar.gz 
INFO[0077] Copying file workspace/Dockerfile to /src/Dockerfile 
INFO[0077] Copying file workspace/Makefile to /src/Makefile 
INFO[0077] Copying file workspace/README.md to /src/README.md 
INFO[0077] Creating directory /src/archetypes           
INFO[0077] Copying file workspace/archetypes/default.md to /src/archetypes/default.md 
INFO[0077] Copying file workspace/codefresh-master.yml to /src/codefresh-master.yml 
INFO[0077] Copying file workspace/config.toml to /src/config.toml 
INFO[0077] Creating directory /src/content              
INFO[0077] Copying file workspace/content/.DS_Store to /src/content/.DS_Store 
INFO[0077] Creating directory /src/content/img          
INFO[0077] Copying file workspace/content/img/.DS_Store to /src/content/img/.DS_Store 
INFO[0077] Copying file workspace/content/img/banner.jpg to /src/content/img/banner.jpg 
INFO[0077] Copying file workspace/content/img/canary-small.jpg to /src/content/img/canary-small.jpg 
INFO[0077] Copying file workspace/content/img/canary-smaller.jpg to /src/content/img/canary-smaller.jpg 
INFO[0077] Copying file workspace/content/img/catalog-small.jpg to /src/content/img/catalog-small.jpg 
INFO[0077] Copying file workspace/content/img/catalog-smaller.jpg to /src/content/img/catalog-smaller.jpg 
INFO[0077] Copying file workspace/content/img/chaos-small.jpg to /src/content/img/chaos-small.jpg 
INFO[0077] Copying file workspace/content/img/chaos-smaller.jpg to /src/content/img/chaos-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops20-small.jpg to /src/content/img/devops20-small.jpg 
INFO[0077] Copying file workspace/content/img/devops20-smaller.jpg to /src/content/img/devops20-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops21-small.png to /src/content/img/devops21-small.png 
INFO[0077] Copying file workspace/content/img/devops21-smaller.png to /src/content/img/devops21-smaller.png 
INFO[0077] Copying file workspace/content/img/devops22-small.jpg to /src/content/img/devops22-small.jpg 
INFO[0077] Copying file workspace/content/img/devops22-smaller.jpg to /src/content/img/devops22-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops23-small.jpg to /src/content/img/devops23-small.jpg 
INFO[0077] Copying file workspace/content/img/devops23-smaller.jpg to /src/content/img/devops23-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops24-small.jpg to /src/content/img/devops24-small.jpg 
INFO[0077] Copying file workspace/content/img/devops24-small.png to /src/content/img/devops24-small.png 
INFO[0077] Copying file workspace/content/img/devops24-smaller.jpg to /src/content/img/devops24-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops24-smaller.png to /src/content/img/devops24-smaller.png 
INFO[0077] Copying file workspace/content/img/devops25-small.jpg to /src/content/img/devops25-small.jpg 
INFO[0077] Copying file workspace/content/img/devops25-small.png to /src/content/img/devops25-small.png 
INFO[0077] Copying file workspace/content/img/devops25-smaller.jpg to /src/content/img/devops25-smaller.jpg 
INFO[0077] Copying file workspace/content/img/devops26-small.jpg to /src/content/img/devops26-small.jpg 
INFO[0077] Copying file workspace/content/img/devops26-smaller.jpg to /src/content/img/devops26-smaller.jpg 
INFO[0077] Creating directory /src/content/posts        
INFO[0077] Copying file workspace/content/posts/canary.md to /src/content/posts/canary.md 
INFO[0077] Copying file workspace/content/posts/catalog.md to /src/content/posts/catalog.md 
INFO[0077] Copying file workspace/content/posts/chaos.md to /src/content/posts/chaos.md 
INFO[0077] Copying file workspace/content/posts/devops-20.md to /src/content/posts/devops-20.md 
INFO[0077] Copying file workspace/content/posts/devops-21.md to /src/content/posts/devops-21.md 
INFO[0077] Copying file workspace/content/posts/devops-22.md to /src/content/posts/devops-22.md 
INFO[0077] Copying file workspace/content/posts/devops-23.md to /src/content/posts/devops-23.md 
INFO[0077] Copying file workspace/content/posts/devops-24.md to /src/content/posts/devops-24.md 
INFO[0077] Copying file workspace/content/posts/devops-25.md to /src/content/posts/devops-25.md 
INFO[0077] Copying file workspace/content/posts/devops-26.md to /src/content/posts/devops-26.md 
INFO[0077] Copying file workspace/docker-socket.yaml to /src/docker-socket.yaml 
INFO[0077] Copying file workspace/docker.yaml to /src/docker.yaml 
INFO[0077] Creating directory /src/etc                  
INFO[0077] Creating directory /src/etc/resolvconf       
INFO[0077] Creating directory /src/etc/resolvconf/resolv.conf.d 
INFO[0077] Copying file workspace/kaniko-dir.yaml to /src/kaniko-dir.yaml 
INFO[0077] Copying file workspace/kaniko-git.yaml to /src/kaniko-git.yaml 
INFO[0077] Copying file workspace/kaniko-git2.yaml to /src/kaniko-git2.yaml 
INFO[0077] Creating directory /src/layouts              
INFO[0077] Creating directory /src/layouts/partials     
INFO[0077] Copying file workspace/layouts/partials/header.html to /src/layouts/partials/header.html 
INFO[0077] Copying file workspace/secret.sh to /src/secret.sh 
INFO[0077] Copying file workspace/shawdowsocks.json to /src/shawdowsocks.json 
INFO[0077] Creating directory /src/static               
INFO[0077] Copying file workspace/static/.DS_Store to /src/static/.DS_Store 
INFO[0077] Creating directory /src/static/css           
INFO[0077] Copying file workspace/static/css/font-awesome.min.css to /src/static/css/font-awesome.min.css 
INFO[0077] Copying file workspace/static/css/ie8.css to /src/static/css/ie8.css 
INFO[0077] Copying file workspace/static/css/ie9.css to /src/static/css/ie9.css 
INFO[0077] Copying file workspace/static/css/main.css to /src/static/css/main.css 
INFO[0077] Creating directory /src/static/css-dimension 
INFO[0077] Copying file workspace/static/css-dimension/bg.jpg to /src/static/css-dimension/bg.jpg 
INFO[0077] Copying file workspace/static/css-dimension/font-awesome.min.css to /src/static/css-dimension/font-awesome.min.css 
INFO[0077] Copying file workspace/static/css-dimension/ie9.css to /src/static/css-dimension/ie9.css 
INFO[0077] Copying file workspace/static/css-dimension/main.css to /src/static/css-dimension/main.css 
INFO[0077] Copying file workspace/static/css-dimension/noscript.css to /src/static/css-dimension/noscript.css 
INFO[0077] Copying file workspace/static/css-dimension/overlay.png to /src/static/css-dimension/overlay.png 
INFO[0077] Copying file workspace/static/css-dimension/project.css to /src/static/css-dimension/project.css 
INFO[0077] Creating directory /src/static/fonts         
INFO[0077] Copying file workspace/static/fonts/FontAwesome.otf to /src/static/fonts/FontAwesome.otf 
INFO[0077] Copying file workspace/static/fonts/fontawesome-webfont.eot to /src/static/fonts/fontawesome-webfont.eot 
INFO[0077] Copying file workspace/static/fonts/fontawesome-webfont.svg to /src/static/fonts/fontawesome-webfont.svg 
INFO[0077] Copying file workspace/static/fonts/fontawesome-webfont.ttf to /src/static/fonts/fontawesome-webfont.ttf 
INFO[0077] Copying file workspace/static/fonts/fontawesome-webfont.woff to /src/static/fonts/fontawesome-webfont.woff 
INFO[0077] Copying file workspace/static/fonts/fontawesome-webfont.woff2 to /src/static/fonts/fontawesome-webfont.woff2 
INFO[0077] Creating directory /src/static/fonts-dimension 
INFO[0077] Copying file workspace/static/fonts-dimension/FontAwesome.otf to /src/static/fonts-dimension/FontAwesome.otf 
INFO[0077] Copying file workspace/static/fonts-dimension/fontawesome-webfont.eot to /src/static/fonts-dimension/fontawesome-webfont.eot 
INFO[0077] Copying file workspace/static/fonts-dimension/fontawesome-webfont.svg to /src/static/fonts-dimension/fontawesome-webfont.svg 
INFO[0077] Copying file workspace/static/fonts-dimension/fontawesome-webfont.ttf to /src/static/fonts-dimension/fontawesome-webfont.ttf 
INFO[0077] Copying file workspace/static/fonts-dimension/fontawesome-webfont.woff to /src/static/fonts-dimension/fontawesome-webfont.woff 
INFO[0077] Copying file workspace/static/fonts-dimension/fontawesome-webfont.woff2 to /src/static/fonts-dimension/fontawesome-webfont.woff2 
INFO[0077] Creating directory /src/static/images        
INFO[0077] Copying file workspace/static/images/devops22-small.jpg to /src/static/images/devops22-small.jpg 
INFO[0077] Copying file workspace/static/images/devops22.jpg to /src/static/images/devops22.jpg 
INFO[0077] Copying file workspace/static/images/devops23-small.jpg to /src/static/images/devops23-small.jpg 
INFO[0077] Copying file workspace/static/images/viktor.png to /src/static/images/viktor.png 
INFO[0077] Creating directory /src/static/js            
INFO[0077] Creating directory /src/static/js/ie         
INFO[0077] Copying file workspace/static/js/ie/backgroundsize.min.htc to /src/static/js/ie/backgroundsize.min.htc 
INFO[0077] Copying file workspace/static/js/ie/html5shiv.js to /src/static/js/ie/html5shiv.js 
INFO[0077] Copying file workspace/static/js/ie/respond.min.js to /src/static/js/ie/respond.min.js 
INFO[0077] Copying file workspace/static/js/jquery.min.js to /src/static/js/jquery.min.js 
INFO[0077] Copying file workspace/static/js/jquery.scrollex.min.js to /src/static/js/jquery.scrollex.min.js 
INFO[0077] Copying file workspace/static/js/jquery.scrolly.min.js to /src/static/js/jquery.scrolly.min.js 
INFO[0077] Copying file workspace/static/js/main.js to /src/static/js/main.js 
INFO[0077] Copying file workspace/static/js/skel.min.js to /src/static/js/skel.min.js 
INFO[0077] Copying file workspace/static/js/util.js to /src/static/js/util.js 
INFO[0077] Creating directory /src/static/js-dimension  
INFO[0077] Copying file workspace/static/js-dimension/jquery.min.js to /src/static/js-dimension/jquery.min.js 
INFO[0077] Copying file workspace/static/js-dimension/main.js to /src/static/js-dimension/main.js 
INFO[0077] Copying file workspace/static/js-dimension/skel.min.js to /src/static/js-dimension/skel.min.js 
INFO[0077] Copying file workspace/static/js-dimension/util.js to /src/static/js-dimension/util.js 
INFO[0077] Creating directory /src/static/sass          
INFO[0077] Creating directory /src/static/sass/base     
INFO[0077] Copying file workspace/static/sass/base/_page.scss to /src/static/sass/base/_page.scss 
INFO[0077] Copying file workspace/static/sass/base/_typography.scss to /src/static/sass/base/_typography.scss 
INFO[0077] Creating directory /src/static/sass/components 
INFO[0077] Copying file workspace/static/sass/components/_box.scss to /src/static/sass/components/_box.scss 
INFO[0077] Copying file workspace/static/sass/components/_button.scss to /src/static/sass/components/_button.scss 
INFO[0077] Copying file workspace/static/sass/components/_contact-method.scss to /src/static/sass/components/_contact-method.scss 
INFO[0077] Copying file workspace/static/sass/components/_form.scss to /src/static/sass/components/_form.scss 
INFO[0077] Copying file workspace/static/sass/components/_icon.scss to /src/static/sass/components/_icon.scss 
INFO[0077] Copying file workspace/static/sass/components/_image.scss to /src/static/sass/components/_image.scss 
INFO[0077] Copying file workspace/static/sass/components/_list.scss to /src/static/sass/components/_list.scss 
INFO[0077] Copying file workspace/static/sass/components/_section.scss to /src/static/sass/components/_section.scss 
INFO[0077] Copying file workspace/static/sass/components/_spotlights.scss to /src/static/sass/components/_spotlights.scss 
INFO[0077] Copying file workspace/static/sass/components/_table.scss to /src/static/sass/components/_table.scss 
INFO[0077] Copying file workspace/static/sass/components/_tiles.scss to /src/static/sass/components/_tiles.scss 
INFO[0077] Copying file workspace/static/sass/ie8.scss to /src/static/sass/ie8.scss 
INFO[0077] Copying file workspace/static/sass/ie9.scss to /src/static/sass/ie9.scss 
INFO[0077] Creating directory /src/static/sass/layout   
INFO[0077] Copying file workspace/static/sass/layout/_banner.scss to /src/static/sass/layout/_banner.scss 
INFO[0077] Copying file workspace/static/sass/layout/_contact.scss to /src/static/sass/layout/_contact.scss 
INFO[0077] Copying file workspace/static/sass/layout/_footer.scss to /src/static/sass/layout/_footer.scss 
INFO[0077] Copying file workspace/static/sass/layout/_header.scss to /src/static/sass/layout/_header.scss 
INFO[0077] Copying file workspace/static/sass/layout/_main.scss to /src/static/sass/layout/_main.scss 
INFO[0077] Copying file workspace/static/sass/layout/_menu.scss to /src/static/sass/layout/_menu.scss 
INFO[0077] Copying file workspace/static/sass/layout/_wrapper.scss to /src/static/sass/layout/_wrapper.scss 
INFO[0077] Creating directory /src/static/sass/libs     
INFO[0077] Copying file workspace/static/sass/libs/_functions.scss to /src/static/sass/libs/_functions.scss 
INFO[0077] Copying file workspace/static/sass/libs/_mixins.scss to /src/static/sass/libs/_mixins.scss 
INFO[0077] Copying file workspace/static/sass/libs/_skel.scss to /src/static/sass/libs/_skel.scss 
INFO[0077] Copying file workspace/static/sass/libs/_vars.scss to /src/static/sass/libs/_vars.scss 
INFO[0077] Copying file workspace/static/sass/main.scss to /src/static/sass/main.scss 
INFO[0077] Creating directory /src/static/sass-dimension 
INFO[0077] Creating directory /src/static/sass-dimension/base 
INFO[0077] Copying file workspace/static/sass-dimension/base/_page.scss to /src/static/sass-dimension/base/_page.scss 
INFO[0077] Copying file workspace/static/sass-dimension/base/_typography.scss to /src/static/sass-dimension/base/_typography.scss 
INFO[0077] Creating directory /src/static/sass-dimension/components 
INFO[0077] Copying file workspace/static/sass-dimension/components/_box.scss to /src/static/sass-dimension/components/_box.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_button.scss to /src/static/sass-dimension/components/_button.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_form.scss to /src/static/sass-dimension/components/_form.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_icon.scss to /src/static/sass-dimension/components/_icon.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_image.scss to /src/static/sass-dimension/components/_image.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_list.scss to /src/static/sass-dimension/components/_list.scss 
INFO[0077] Copying file workspace/static/sass-dimension/components/_table.scss to /src/static/sass-dimension/components/_table.scss 
INFO[0077] Copying file workspace/static/sass-dimension/ie9.scss to /src/static/sass-dimension/ie9.scss 
INFO[0077] Creating directory /src/static/sass-dimension/layout 
INFO[0077] Copying file workspace/static/sass-dimension/layout/_bg.scss to /src/static/sass-dimension/layout/_bg.scss 
INFO[0077] Copying file workspace/static/sass-dimension/layout/_footer.scss to /src/static/sass-dimension/layout/_footer.scss 
INFO[0077] Copying file workspace/static/sass-dimension/layout/_header.scss to /src/static/sass-dimension/layout/_header.scss 
INFO[0077] Copying file workspace/static/sass-dimension/layout/_main.scss to /src/static/sass-dimension/layout/_main.scss 
INFO[0077] Copying file workspace/static/sass-dimension/layout/_wrapper.scss to /src/static/sass-dimension/layout/_wrapper.scss 
INFO[0077] Creating directory /src/static/sass-dimension/libs 
INFO[0077] Copying file workspace/static/sass-dimension/libs/_functions.scss to /src/static/sass-dimension/libs/_functions.scss 
INFO[0077] Copying file workspace/static/sass-dimension/libs/_mixins.scss to /src/static/sass-dimension/libs/_mixins.scss 
INFO[0077] Copying file workspace/static/sass-dimension/libs/_skel.scss to /src/static/sass-dimension/libs/_skel.scss 
INFO[0077] Copying file workspace/static/sass-dimension/libs/_vars.scss to /src/static/sass-dimension/libs/_vars.scss 
INFO[0077] Copying file workspace/static/sass-dimension/main.scss to /src/static/sass-dimension/main.scss 
INFO[0077] Copying file workspace/static/sass-dimension/noscript.scss to /src/static/sass-dimension/noscript.scss 
INFO[0077] Creating directory /src/themes               
INFO[0077] Copying file workspace/themes/.DS_Store to /src/themes/.DS_Store 
INFO[0077] Creating directory /src/themes/forty         
INFO[0077] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0077] RUN make init                                
INFO[0077] cmd: /bin/sh                                 
INFO[0077] args: [-c make init]                         
git submodule init
Submodule 'themes/forty' (https://github.com/MarcusVirg/forty) registered for path 'themes/forty'
git submodule update
Cloning into '/src/themes/forty'...
Submodule path 'themes/forty': checked out 'dccea57bd2ed194942080d650671b47b6df4183c'
cp content/img/banner.jpg themes/forty/static/img/.
INFO[0089] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0089] RUN make build                               
INFO[0089] cmd: /bin/sh                                 
INFO[0089] args: [-c make build]                        
hugo
Start building sites … 

                   | EN  
-------------------+-----
  Pages            | 19  
  Paginator pages  |  0  
  Non-page files   | 24  
  Static files     | 97  
  Processed images |  0  
  Aliases          |  0  
  Sitemaps         |  1  
  Cleaned          |  0  

Total in 528 ms
INFO[0040] Taking snapshot of full filesystem...        
INFO[0051] Storing source image from stage 0 at path /kaniko/stages/0 
INFO[0079] trying to extract to /kaniko/0               
INFO[0079] Extracting layer 0                           
INFO[0083] Extracting layer 1                           
INFO[0098] Extracting layer 2                           
INFO[0103] Extracting layer 3                           
INFO[0104] Deleting filesystem...                       
INFO[0106] Downloading base image nginx:1.19.4-alpine   
2021/12/08 14:20:59 No matching credentials were found, falling back on anonymous
INFO[0111] Extracting layer 0                           
INFO[0120] Extracting layer 1                           
INFO[0141] Extracting layer 2                           
INFO[0145] Extracting layer 3                           
INFO[0147] Extracting layer 4                           
INFO[0148] Taking snapshot of full filesystem...        
INFO[0152] RUN mv /usr/share/nginx/html/index.html /usr/share/nginx/html/old-index.html 
INFO[0152] cmd: /bin/sh                                 
INFO[0152] args: [-c mv /usr/share/nginx/html/index.html /usr/share/nginx/html/old-index.html] 
INFO[0152] Taking snapshot of full filesystem...        
INFO[0153] Adding whiteout for /usr/share/nginx/html/index.html 
INFO[0154] COPY --from=build /src/public /usr/share/nginx/html 
INFO[0154] Creating directory /usr/share/nginx/html     
INFO[0154] Copying file /kaniko/0/src/public/.DS_Store to /usr/share/nginx/html/.DS_Store 
INFO[0154] Copying file /kaniko/0/src/public/404.html to /usr/share/nginx/html/404.html 
INFO[0154] Creating directory /usr/share/nginx/html/categories 
INFO[0154] Copying file /kaniko/0/src/public/categories/index.html to /usr/share/nginx/html/categories/index.html 
INFO[0154] Copying file /kaniko/0/src/public/categories/index.xml to /usr/share/nginx/html/categories/index.xml 
INFO[0154] Creating directory /usr/share/nginx/html/css 
INFO[0154] Copying file /kaniko/0/src/public/css/font-awesome.min.css to /usr/share/nginx/html/css/font-awesome.min.css 
INFO[0154] Copying file /kaniko/0/src/public/css/ie8.css to /usr/share/nginx/html/css/ie8.css 
INFO[0154] Copying file /kaniko/0/src/public/css/ie9.css to /usr/share/nginx/html/css/ie9.css 
INFO[0154] Copying file /kaniko/0/src/public/css/main.css to /usr/share/nginx/html/css/main.css 
INFO[0154] Creating directory /usr/share/nginx/html/css-dimension 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/bg.jpg to /usr/share/nginx/html/css-dimension/bg.jpg 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/font-awesome.min.css to /usr/share/nginx/html/css-dimension/font-awesome.min.css 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/ie9.css to /usr/share/nginx/html/css-dimension/ie9.css 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/main.css to /usr/share/nginx/html/css-dimension/main.css 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/noscript.css to /usr/share/nginx/html/css-dimension/noscript.css 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/overlay.png to /usr/share/nginx/html/css-dimension/overlay.png 
INFO[0154] Copying file /kaniko/0/src/public/css-dimension/project.css to /usr/share/nginx/html/css-dimension/project.css 
INFO[0154] Copying file /kaniko/0/src/public/elements.html to /usr/share/nginx/html/elements.html 
INFO[0154] Creating directory /usr/share/nginx/html/fonts 
INFO[0154] Copying file /kaniko/0/src/public/fonts/FontAwesome.otf to /usr/share/nginx/html/fonts/FontAwesome.otf 
INFO[0154] Copying file /kaniko/0/src/public/fonts/fontawesome-webfont.eot to /usr/share/nginx/html/fonts/fontawesome-webfont.eot 
INFO[0154] Copying file /kaniko/0/src/public/fonts/fontawesome-webfont.svg to /usr/share/nginx/html/fonts/fontawesome-webfont.svg 
INFO[0154] Copying file /kaniko/0/src/public/fonts/fontawesome-webfont.ttf to /usr/share/nginx/html/fonts/fontawesome-webfont.ttf 
INFO[0154] Copying file /kaniko/0/src/public/fonts/fontawesome-webfont.woff to /usr/share/nginx/html/fonts/fontawesome-webfont.woff 
INFO[0154] Copying file /kaniko/0/src/public/fonts/fontawesome-webfont.woff2 to /usr/share/nginx/html/fonts/fontawesome-webfont.woff2 
INFO[0154] Creating directory /usr/share/nginx/html/fonts-dimension 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/FontAwesome.otf to /usr/share/nginx/html/fonts-dimension/FontAwesome.otf 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/fontawesome-webfont.eot to /usr/share/nginx/html/fonts-dimension/fontawesome-webfont.eot 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/fontawesome-webfont.svg to /usr/share/nginx/html/fonts-dimension/fontawesome-webfont.svg 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/fontawesome-webfont.ttf to /usr/share/nginx/html/fonts-dimension/fontawesome-webfont.ttf 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/fontawesome-webfont.woff to /usr/share/nginx/html/fonts-dimension/fontawesome-webfont.woff 
INFO[0154] Copying file /kaniko/0/src/public/fonts-dimension/fontawesome-webfont.woff2 to /usr/share/nginx/html/fonts-dimension/fontawesome-webfont.woff2 
INFO[0154] Creating directory /usr/share/nginx/html/images 
INFO[0154] Copying file /kaniko/0/src/public/images/devops22-small.jpg to /usr/share/nginx/html/images/devops22-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/images/devops22.jpg to /usr/share/nginx/html/images/devops22.jpg 
INFO[0154] Copying file /kaniko/0/src/public/images/devops23-small.jpg to /usr/share/nginx/html/images/devops23-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/images/viktor.png to /usr/share/nginx/html/images/viktor.png 
INFO[0154] Creating directory /usr/share/nginx/html/img 
INFO[0154] Copying file /kaniko/0/src/public/img/banner.jpg to /usr/share/nginx/html/img/banner.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/canary-small.jpg to /usr/share/nginx/html/img/canary-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/canary-smaller.jpg to /usr/share/nginx/html/img/canary-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/catalog-small.jpg to /usr/share/nginx/html/img/catalog-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/catalog-smaller.jpg to /usr/share/nginx/html/img/catalog-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/chaos-small.jpg to /usr/share/nginx/html/img/chaos-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/chaos-smaller.jpg to /usr/share/nginx/html/img/chaos-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops20-small.jpg to /usr/share/nginx/html/img/devops20-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops20-smaller.jpg to /usr/share/nginx/html/img/devops20-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops21-small.png to /usr/share/nginx/html/img/devops21-small.png 
INFO[0154] Copying file /kaniko/0/src/public/img/devops21-smaller.png to /usr/share/nginx/html/img/devops21-smaller.png 
INFO[0154] Copying file /kaniko/0/src/public/img/devops22-small.jpg to /usr/share/nginx/html/img/devops22-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops22-smaller.jpg to /usr/share/nginx/html/img/devops22-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops23-small.jpg to /usr/share/nginx/html/img/devops23-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops23-smaller.jpg to /usr/share/nginx/html/img/devops23-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops24-small.jpg to /usr/share/nginx/html/img/devops24-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops24-small.png to /usr/share/nginx/html/img/devops24-small.png 
INFO[0154] Copying file /kaniko/0/src/public/img/devops24-smaller.jpg to /usr/share/nginx/html/img/devops24-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops24-smaller.png to /usr/share/nginx/html/img/devops24-smaller.png 
INFO[0154] Copying file /kaniko/0/src/public/img/devops25-small.jpg to /usr/share/nginx/html/img/devops25-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops25-small.png to /usr/share/nginx/html/img/devops25-small.png 
INFO[0154] Copying file /kaniko/0/src/public/img/devops25-smaller.jpg to /usr/share/nginx/html/img/devops25-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops26-small.jpg to /usr/share/nginx/html/img/devops26-small.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/devops26-smaller.jpg to /usr/share/nginx/html/img/devops26-smaller.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic01.jpg to /usr/share/nginx/html/img/pic01.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic02.jpg to /usr/share/nginx/html/img/pic02.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic03.jpg to /usr/share/nginx/html/img/pic03.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic04.jpg to /usr/share/nginx/html/img/pic04.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic05.jpg to /usr/share/nginx/html/img/pic05.jpg 
INFO[0154] Copying file /kaniko/0/src/public/img/pic06.jpg to /usr/share/nginx/html/img/pic06.jpg 
INFO[0154] Copying file /kaniko/0/src/public/index.html to /usr/share/nginx/html/index.html 
INFO[0154] Copying file /kaniko/0/src/public/index.xml to /usr/share/nginx/html/index.xml 
INFO[0154] Creating directory /usr/share/nginx/html/js  
INFO[0154] Creating directory /usr/share/nginx/html/js/ie 
INFO[0154] Copying file /kaniko/0/src/public/js/ie/backgroundsize.min.htc to /usr/share/nginx/html/js/ie/backgroundsize.min.htc 
INFO[0154] Copying file /kaniko/0/src/public/js/ie/html5shiv.js to /usr/share/nginx/html/js/ie/html5shiv.js 
INFO[0154] Copying file /kaniko/0/src/public/js/ie/respond.min.js to /usr/share/nginx/html/js/ie/respond.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js/jquery.min.js to /usr/share/nginx/html/js/jquery.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js/jquery.scrollex.min.js to /usr/share/nginx/html/js/jquery.scrollex.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js/jquery.scrolly.min.js to /usr/share/nginx/html/js/jquery.scrolly.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js/main.js to /usr/share/nginx/html/js/main.js 
INFO[0154] Copying file /kaniko/0/src/public/js/skel.min.js to /usr/share/nginx/html/js/skel.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js/util.js to /usr/share/nginx/html/js/util.js 
INFO[0154] Creating directory /usr/share/nginx/html/js-dimension 
INFO[0154] Copying file /kaniko/0/src/public/js-dimension/jquery.min.js to /usr/share/nginx/html/js-dimension/jquery.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js-dimension/main.js to /usr/share/nginx/html/js-dimension/main.js 
INFO[0154] Copying file /kaniko/0/src/public/js-dimension/skel.min.js to /usr/share/nginx/html/js-dimension/skel.min.js 
INFO[0154] Copying file /kaniko/0/src/public/js-dimension/util.js to /usr/share/nginx/html/js-dimension/util.js 
INFO[0154] Creating directory /usr/share/nginx/html/posts 
INFO[0154] Creating directory /usr/share/nginx/html/posts/canary 
INFO[0154] Copying file /kaniko/0/src/public/posts/canary/index.html to /usr/share/nginx/html/posts/canary/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/catalog 
INFO[0154] Copying file /kaniko/0/src/public/posts/catalog/index.html to /usr/share/nginx/html/posts/catalog/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/chaos 
INFO[0154] Copying file /kaniko/0/src/public/posts/chaos/index.html to /usr/share/nginx/html/posts/chaos/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-20 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-20/index.html to /usr/share/nginx/html/posts/devops-20/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-21 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-21/index.html to /usr/share/nginx/html/posts/devops-21/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-22 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-22/index.html to /usr/share/nginx/html/posts/devops-22/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-23 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-23/index.html to /usr/share/nginx/html/posts/devops-23/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-24 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-24/index.html to /usr/share/nginx/html/posts/devops-24/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-25 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-25/index.html to /usr/share/nginx/html/posts/devops-25/index.html 
INFO[0154] Creating directory /usr/share/nginx/html/posts/devops-26 
INFO[0154] Copying file /kaniko/0/src/public/posts/devops-26/index.html to /usr/share/nginx/html/posts/devops-26/index.html 
INFO[0154] Copying file /kaniko/0/src/public/posts/index.html to /usr/share/nginx/html/posts/index.html 
INFO[0154] Copying file /kaniko/0/src/public/posts/index.xml to /usr/share/nginx/html/posts/index.xml 
INFO[0154] Creating directory /usr/share/nginx/html/sass 
INFO[0154] Creating directory /usr/share/nginx/html/sass/base 
INFO[0154] Copying file /kaniko/0/src/public/sass/base/_page.scss to /usr/share/nginx/html/sass/base/_page.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/base/_typography.scss to /usr/share/nginx/html/sass/base/_typography.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass/components 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_box.scss to /usr/share/nginx/html/sass/components/_box.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_button.scss to /usr/share/nginx/html/sass/components/_button.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_contact-method.scss to /usr/share/nginx/html/sass/components/_contact-method.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_form.scss to /usr/share/nginx/html/sass/components/_form.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_icon.scss to /usr/share/nginx/html/sass/components/_icon.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_image.scss to /usr/share/nginx/html/sass/components/_image.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_list.scss to /usr/share/nginx/html/sass/components/_list.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_section.scss to /usr/share/nginx/html/sass/components/_section.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_spotlights.scss to /usr/share/nginx/html/sass/components/_spotlights.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_table.scss to /usr/share/nginx/html/sass/components/_table.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/components/_tiles.scss to /usr/share/nginx/html/sass/components/_tiles.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/ie8.scss to /usr/share/nginx/html/sass/ie8.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/ie9.scss to /usr/share/nginx/html/sass/ie9.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass/layout 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_banner.scss to /usr/share/nginx/html/sass/layout/_banner.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_contact.scss to /usr/share/nginx/html/sass/layout/_contact.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_footer.scss to /usr/share/nginx/html/sass/layout/_footer.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_header.scss to /usr/share/nginx/html/sass/layout/_header.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_main.scss to /usr/share/nginx/html/sass/layout/_main.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_menu.scss to /usr/share/nginx/html/sass/layout/_menu.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/layout/_wrapper.scss to /usr/share/nginx/html/sass/layout/_wrapper.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass/libs 
INFO[0154] Copying file /kaniko/0/src/public/sass/libs/_functions.scss to /usr/share/nginx/html/sass/libs/_functions.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/libs/_mixins.scss to /usr/share/nginx/html/sass/libs/_mixins.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/libs/_skel.scss to /usr/share/nginx/html/sass/libs/_skel.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/libs/_vars.scss to /usr/share/nginx/html/sass/libs/_vars.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass/main.scss to /usr/share/nginx/html/sass/main.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass-dimension 
INFO[0154] Creating directory /usr/share/nginx/html/sass-dimension/base 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/base/_page.scss to /usr/share/nginx/html/sass-dimension/base/_page.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/base/_typography.scss to /usr/share/nginx/html/sass-dimension/base/_typography.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass-dimension/components 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_box.scss to /usr/share/nginx/html/sass-dimension/components/_box.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_button.scss to /usr/share/nginx/html/sass-dimension/components/_button.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_form.scss to /usr/share/nginx/html/sass-dimension/components/_form.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_icon.scss to /usr/share/nginx/html/sass-dimension/components/_icon.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_image.scss to /usr/share/nginx/html/sass-dimension/components/_image.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_list.scss to /usr/share/nginx/html/sass-dimension/components/_list.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/components/_table.scss to /usr/share/nginx/html/sass-dimension/components/_table.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/ie9.scss to /usr/share/nginx/html/sass-dimension/ie9.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass-dimension/layout 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/layout/_bg.scss to /usr/share/nginx/html/sass-dimension/layout/_bg.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/layout/_footer.scss to /usr/share/nginx/html/sass-dimension/layout/_footer.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/layout/_header.scss to /usr/share/nginx/html/sass-dimension/layout/_header.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/layout/_main.scss to /usr/share/nginx/html/sass-dimension/layout/_main.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/layout/_wrapper.scss to /usr/share/nginx/html/sass-dimension/layout/_wrapper.scss 
INFO[0154] Creating directory /usr/share/nginx/html/sass-dimension/libs 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/libs/_functions.scss to /usr/share/nginx/html/sass-dimension/libs/_functions.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/libs/_mixins.scss to /usr/share/nginx/html/sass-dimension/libs/_mixins.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/libs/_skel.scss to /usr/share/nginx/html/sass-dimension/libs/_skel.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/libs/_vars.scss to /usr/share/nginx/html/sass-dimension/libs/_vars.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/main.scss to /usr/share/nginx/html/sass-dimension/main.scss 
INFO[0154] Copying file /kaniko/0/src/public/sass-dimension/noscript.scss to /usr/share/nginx/html/sass-dimension/noscript.scss 
INFO[0154] Copying file /kaniko/0/src/public/sitemap.xml to /usr/share/nginx/html/sitemap.xml 
INFO[0154] Creating directory /usr/share/nginx/html/tags 
INFO[0154] Copying file /kaniko/0/src/public/tags/index.html to /usr/share/nginx/html/tags/index.html 
INFO[0154] Copying file /kaniko/0/src/public/tags/index.xml to /usr/share/nginx/html/tags/index.xml 
INFO[0154] Taking snapshot of files...                  
INFO[0155] EXPOSE 80                                    
INFO[0155] cmd: EXPOSE                                  
INFO[0155] Adding exposed port: 80/tcp                  
INFO[0155] No files changed in this command, skipping snapshotting. 
INFO[0155] No files were changed, appending empty layer to config. No layer added to image. 
error pushing image: failed to push to destination index.docker.io/zongxun/devops-toolkit:1.0.0: unsupported status code 401; body: