Kaniko fails with containerd runtime in GKE

[jeunii]

Actual behavior Running a kaniko build in a GKE cluster with containerd runtime results in the error

standard_init_linux.go:211: exec user process caused "permission denied"

OS selected is Container-Optimized OS with Containerd (cos_containerd).

This happens with the nodes have the following runtime

 Kernel Version:             4.14.138+
 OS Image:                   Container-Optimized OS from Google
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  containerd://1.2.8

Expected behavior The build should start successfully.

This does happen when the runtime is docker

 Kernel Version:             4.14.138+
 OS Image:                   Container-Optimized OS from Google
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://18.9.7

OS selected is Container-Optimized OS (cos)

To Reproduce Launch a build job on a node that uses containerd runtime instead of docker. Use the gce.privileged PSP and run the container as privileged.

Additional Information

• Dockerfile Please provide either the Dockerfile you're trying to build or one that can reproduce this error.
• Build Context Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands)

• Kaniko Image (fully qualified with digest)

  debug-v0.17.1

  Triage Notes for the Maintainers

  Description	Yes/No
  Please check if this a new feature you are proposing	- [ ]
  Please check if the build works in docker but not in kaniko	- [ X]
  Please check if this error is seen when you use --cache flag	- [ ]
  Please check if your dockerfile is a multistage dockerfile	- [X ]

[vguaglione]

We have a similar problem using the kaniko container inside the kubernetes gitlab-runner deployed in an OpenShift 3.11 cluster. Our build works fine when running on a standard runner VM or on the local workstations, but fails inside a cluster.

Our Dockerfile:

FROM node:10-alpine

RUN export SHELL=/bin/sh && \ npm upgrade -g

RUN npm list -g RUN apk upgrade -U

RUN npm set registry https://npm.dhe.duke.edu && \ npm install -g \ eslint eslint-plugin-html \ mocha c8 nyc \ link-checker \ swagger-tools \
newman@4 \ sinon@1 \ markdown-spellcheck

RUN npm list -g --depth=1 RUN apk upgrade -U

CMD /bin/sh

The error we see:

INFO[0000] Resolved base name node:10-alpine to node:10-alpine INFO[0000] Resolved base name node:10-alpine to node:10-alpine INFO[0000] Retrieving image manifest node:10-alpine
INFO[0000] Retrieving image manifest node:10-alpine
INFO[0001] Built cross stage deps: map[]
INFO[0001] Retrieving image manifest node:10-alpine
INFO[0001] Retrieving image manifest node:10-alpine
INFO[0001] Unpacking rootfs as cmd RUN export SHELL=/bin/sh && npm upgrade -g requires it. INFO[0003] Taking snapshot of full filesystem...
INFO[0004] Resolving paths
INFO[0004] RUN export SHELL=/bin/sh && npm upgrade -g INFO[0004] cmd: /bin/sh
INFO[0004] args: [-c export SHELL=/bin/sh && npm upgrade -g] INFO[0005] Taking snapshot of full filesystem...
INFO[0006] Resolving paths
INFO[0006] RUN npm list -g
INFO[0006] cmd: /bin/sh
INFO[0006] args: [-c npm list -g]
/usr/local/lib -- npm@6.14.4 +-- abbrev@1.1.1 +-- ansicolors@0.3.2 +-- ansistyles@0.1.3 +-- aproba@2.0.0 +-- archy@1.0.0 +-- bin-links@1.1.7 | +-- bluebird@3.5.5 deduped | +-- cmd-shim@3.0.3 deduped | +-- gentle-fs@2.3.0 deduped | +-- graceful-fs@4.2.3 deduped | +-- npm-normalize-package-bin@1.0.1 |-- write-file-atomic@2.4.3 deduped +-- bluebird@3.5.5 +-- byte-size@5.0.1 +-- cacache@12.0.3 | +-- bluebird@3.5.5 deduped | +-- chownr@1.1.4 deduped | +-- figgy-pudding@3.5.1 deduped | +-- glob@7.1.6 deduped | +-- graceful-fs@4.2.3 deduped | +-- infer-owner@1.0.4 deduped | +-- lru-cache@5.1.1 deduped | +-- mississippi@3.0.0 deduped | +-- mkdirp@0.5.4 deduped | +-- move-concurrently@1.0.1 deduped | +-- promise-inflight@1.0.1 deduped | +-- rimraf@2.7.1 deduped | +-- ssri@6.0.1 deduped | +-- unique-filename@1.1.1 deduped | -- y18n@4.0.0 +-- call-limit@1.1.1 +-- chownr@1.1.4 +-- ci-info@2.0.0 +-- cli-columns@3.1.2 | +-- string-width@2.1.1 | | +-- is-fullwidth-code-point@2.0.0 | |-- strip-ansi@4.0.0 | | -- ansi-regex@3.0.0 |-- strip-ansi@3.0.1 | -- ansi-regex@2.1.1 +-- cli-table3@0.5.1 | +-- colors@1.3.3 | +-- object-assign@4.1.1 |-- string-width@2.1.1 deduped +-- cmd-shim@3.0.3 | +-- graceful-fs@4.2.3 deduped | -- mkdirp@0.5.4 deduped +-- columnify@1.5.4 | +-- strip-ansi@3.0.1 deduped |-- wcwidth@1.0.1 | -- defaults@1.0.3 |-- clone@1.0.4 +-- config-chain@1.1.12 | +-- ini@1.3.5 deduped | -- proto-list@1.2.4 +-- debuglog@1.0.1 +-- detect-indent@5.0.0 +-- detect-newline@2.1.0 +-- dezalgo@1.0.3 | +-- asap@2.0.6 |-- wrappy@1.0.2 +-- editor@1.0.0 +-- figgy-pudding@3.5.1 +-- find-npm-prefix@1.0.2 +-- fs-vacuum@1.2.10 | +-- graceful-fs@4.2.3 deduped | +-- path-is-inside@1.0.2 deduped | -- rimraf@2.7.1 deduped +-- fs-write-stream-atomic@1.0.10 | +-- graceful-fs@4.2.3 deduped | +-- iferr@0.1.5 | +-- imurmurhash@0.1.4 deduped |-- readable-stream@2.3.6 | +-- core-util-is@1.0.2 | +-- inherits@2.0.4 deduped | +-- isarray@1.0.0 | +-- process-nextick-args@2.0.0 | +-- safe-buffer@5.1.2 deduped | +-- string_decoder@1.1.1 | | -- safe-buffer@5.1.2 deduped |-- util-deprecate@1.0.2 deduped +-- gentle-fs@2.3.0 | +-- aproba@1.2.0 | +-- chownr@1.1.4 deduped | +-- cmd-shim@3.0.3 deduped | +-- fs-vacuum@1.2.10 deduped | +-- graceful-fs@4.2.3 deduped | +-- iferr@0.1.5 | +-- infer-owner@1.0.4 deduped | +-- mkdirp@0.5.4 deduped | +-- path-is-inside@1.0.2 deduped | +-- read-cmd-shim@1.0.5 deduped | -- slide@1.1.6 deduped +-- glob@7.1.6 | +-- fs.realpath@1.0.0 | +-- inflight@1.0.6 deduped | +-- inherits@2.0.4 deduped | +-- minimatch@3.0.4 | |-- brace-expansion@1.1.11 | | +-- balanced-match@1.0.0 | | -- concat-map@0.0.1 | +-- once@1.4.0 deduped |-- path-is-absolute@1.0.1 +-- graceful-fs@4.2.3 +-- has-unicode@2.0.1 +-- hosted-git-info@2.8.8 +-- iferr@1.0.2 +-- imurmurhash@0.1.4 +-- infer-owner@1.0.4 +-- inflight@1.0.6 | +-- once@1.4.0 deduped | -- wrappy@1.0.2 deduped +-- inherits@2.0.4 +-- ini@1.3.5 +-- init-package-json@1.10.3 | +-- glob@7.1.6 deduped | +-- npm-package-arg@6.1.1 deduped | +-- promzard@0.3.0 | |-- read@1.0.7 deduped | +-- read@1.0.7 deduped | +-- read-package-json@2.1.1 deduped | +-- semver@5.7.1 deduped | +-- validate-npm-package-license@3.0.4 deduped | -- validate-npm-package-name@3.0.0 deduped +-- is-cidr@3.0.0 |-- cidr-regex@2.0.10 | -- ip-regex@2.1.0 +-- json-parse-better-errors@1.0.2 +-- JSONStream@1.3.5 | +-- jsonparse@1.3.1 |-- through@2.3.8 +-- lazy-property@1.0.0 +-- libcipm@4.0.7 | +-- bin-links@1.1.7 deduped | +-- bluebird@3.5.5 deduped | +-- figgy-pudding@3.5.1 deduped | +-- find-npm-prefix@1.0.2 deduped | +-- graceful-fs@4.2.3 deduped | +-- ini@1.3.5 deduped | +-- lock-verify@2.1.0 deduped | +-- mkdirp@0.5.4 deduped | +-- npm-lifecycle@3.1.4 deduped | +-- npm-logical-tree@1.2.1 | +-- npm-package-arg@6.1.1 deduped | +-- pacote@9.5.12 deduped | +-- read-package-json@2.1.1 deduped | +-- rimraf@2.7.1 deduped | -- worker-farm@1.7.0 deduped +-- libnpm@3.0.1 | +-- bin-links@1.1.7 deduped | +-- bluebird@3.5.5 deduped | +-- find-npm-prefix@1.0.2 deduped | +-- libnpmaccess@3.0.2 deduped | +-- libnpmconfig@1.2.1 | | +-- figgy-pudding@3.5.1 deduped | | +-- find-up@3.0.0 | | |-- locate-path@3.0.0 | | | +-- p-locate@3.0.0 | | | | -- p-limit@2.2.0 | | | |-- p-try@2.2.0 | | | -- path-exists@3.0.0 | |-- ini@1.3.5 deduped | +-- libnpmhook@5.0.3 deduped | +-- libnpmorg@1.0.1 deduped | +-- libnpmpublish@1.1.2 | | +-- aproba@2.0.0 deduped | | +-- figgy-pudding@3.5.1 deduped | | +-- get-stream@4.1.0 deduped | | +-- lodash.clonedeep@4.5.0 deduped | | +-- normalize-package-data@2.5.0 deduped | | +-- npm-package-arg@6.1.1 deduped | | +-- npm-registry-fetch@4.0.3 deduped | | +-- semver@5.7.1 deduped | | -- ssri@6.0.1 deduped | +-- libnpmsearch@2.0.2 deduped | +-- libnpmteam@1.0.2 deduped | +-- lock-verify@2.1.0 deduped | +-- npm-lifecycle@3.1.4 deduped | +-- npm-logical-tree@1.2.1 deduped | +-- npm-package-arg@6.1.1 deduped | +-- npm-profile@4.0.4 deduped | +-- npm-registry-fetch@4.0.3 deduped | +-- npmlog@4.1.2 deduped | +-- pacote@9.5.12 deduped | +-- read-package-json@2.1.1 deduped |-- stringify-package@1.0.1 deduped +-- libnpmaccess@3.0.2 | +-- aproba@2.0.0 deduped | +-- get-stream@4.1.0 | | -- pump@3.0.0 deduped | +-- npm-package-arg@6.1.1 deduped |-- npm-registry-fetch@4.0.3 deduped +-- libnpmhook@5.0.3 | +-- aproba@2.0.0 deduped | +-- figgy-pudding@3.5.1 deduped | +-- get-stream@4.1.0 deduped | -- npm-registry-fetch@4.0.3 deduped +-- libnpmorg@1.0.1 | +-- aproba@2.0.0 deduped | +-- figgy-pudding@3.5.1 deduped | +-- get-stream@4.1.0 deduped |-- npm-registry-fetch@4.0.3 deduped +-- libnpmsearch@2.0.2 | +-- figgy-pudding@3.5.1 deduped | +-- get-stream@4.1.0 deduped | -- npm-registry-fetch@4.0.3 deduped +-- libnpmteam@1.0.2 | +-- aproba@2.0.0 deduped | +-- figgy-pudding@3.5.1 deduped | +-- get-stream@4.1.0 deduped |-- npm-registry-fetch@4.0.3 deduped +-- libnpx@10.2.2 | +-- dotenv@5.0.1 | +-- npm-package-arg@6.1.1 deduped | +-- rimraf@2.7.1 deduped | +-- safe-buffer@5.1.2 deduped | +-- update-notifier@2.5.0 deduped | +-- which@1.3.1 deduped | +-- y18n@4.0.0 deduped | -- yargs@11.1.1 | +-- cliui@4.1.0 | | +-- string-width@2.1.1 deduped | | +-- strip-ansi@4.0.0 | | |-- ansi-regex@3.0.0 | | -- wrap-ansi@2.1.0 | | +-- string-width@1.0.2 | | | +-- code-point-at@1.1.0 deduped | | | +-- is-fullwidth-code-point@1.0.0 deduped | | |-- strip-ansi@3.0.1 deduped | | -- strip-ansi@3.0.1 deduped | +-- decamelize@1.2.0 | +-- find-up@2.1.0 | |-- locate-path@2.0.0 | | +-- p-locate@2.0.0 | | | -- p-limit@1.2.0 | | |-- p-try@1.0.0 | | -- path-exists@3.0.0 deduped | +-- get-caller-file@1.0.3 | +-- os-locale@3.1.0 | | +-- execa@1.0.0 | | | +-- cross-spawn@6.0.5 | | | | +-- nice-try@1.0.5 | | | | +-- path-key@2.0.1 deduped | | | | +-- semver@5.7.1 deduped | | | | +-- shebang-command@1.2.0 | | | | |-- shebang-regex@1.0.0 | | | | -- which@1.3.1 deduped | | | +-- get-stream@4.1.0 deduped | | | +-- is-stream@1.1.0 | | | +-- npm-run-path@2.0.2 | | | |-- path-key@2.0.1 | | | +-- p-finally@1.0.0 | | | +-- signal-exit@3.0.2 deduped | | | -- strip-eof@1.0.0 | | +-- lcid@2.0.0 | | |-- invert-kv@2.0.0 | | -- mem@4.3.0 | | +-- map-age-cleaner@0.1.3 | | |-- p-defer@1.0.0 | | +-- mimic-fn@2.1.0 | | -- p-is-promise@2.1.0 | +-- require-directory@2.1.1 | +-- require-main-filename@1.0.1 | +-- set-blocking@2.0.0 deduped | +-- string-width@2.1.1 deduped | +-- which-module@2.0.0 | +-- y18n@3.2.1 |-- yargs-parser@9.0.2 | -- camelcase@4.1.0 deduped +-- lock-verify@2.1.0 | +-- npm-package-arg@6.1.1 deduped |-- semver@5.7.1 deduped +-- lockfile@1.0.4 | -- signal-exit@3.0.2 +-- lodash._baseindexof@3.1.0 +-- lodash._baseuniq@4.6.0 | +-- lodash._createset@4.0.3 |-- lodash._root@3.0.1 +-- lodash._bindcallback@3.0.1 +-- lodash._cacheindexof@3.0.2 +-- lodash._createcache@3.1.2 | -- lodash._getnative@3.9.1 deduped +-- lodash._getnative@3.9.1 +-- lodash.clonedeep@4.5.0 +-- lodash.restparam@3.6.1 +-- lodash.union@4.6.0 +-- lodash.uniq@4.5.0 +-- lodash.without@4.4.0 +-- lru-cache@5.1.1 |-- yallist@3.0.3 +-- meant@1.0.1 +-- mississippi@3.0.0 | +-- concat-stream@1.6.2 | | +-- buffer-from@1.0.0 | | +-- inherits@2.0.4 deduped | | +-- readable-stream@2.3.6 | | | +-- core-util-is@1.0.2 deduped | | | +-- inherits@2.0.4 deduped | | | +-- isarray@1.0.0 deduped | | | +-- process-nextick-args@2.0.0 deduped | | | +-- safe-buffer@5.1.2 deduped | | | +-- string_decoder@1.1.1 | | | | -- safe-buffer@5.1.2 deduped | | |-- util-deprecate@1.0.2 deduped | | -- typedarray@0.0.6 | +-- duplexify@3.6.0 | | +-- end-of-stream@1.4.1 deduped | | +-- inherits@2.0.4 deduped | | +-- readable-stream@2.3.6 | | | +-- core-util-is@1.0.2 deduped | | | +-- inherits@2.0.4 deduped | | | +-- isarray@1.0.0 deduped | | | +-- process-nextick-args@2.0.0 deduped | | | +-- safe-buffer@5.1.2 deduped | | | +-- string_decoder@1.1.1 | | | |-- safe-buffer@5.1.2 deduped | | | -- util-deprecate@1.0.2 deduped | |-- stream-shift@1.0.0 | +-- end-of-stream@1.4.1 | | -- once@1.4.0 deduped | +-- flush-write-stream@1.0.3 | | +-- inherits@2.0.4 deduped | |-- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | | -- safe-buffer@5.1.2 deduped | |-- util-deprecate@1.0.2 deduped | +-- from2@2.3.0 | | +-- inherits@2.0.4 deduped | | -- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | |-- safe-buffer@5.1.2 deduped | | -- util-deprecate@1.0.2 deduped | +-- parallel-transform@1.1.0 | | +-- cyclist@0.2.2 | | +-- inherits@2.0.4 deduped | |-- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | | -- safe-buffer@5.1.2 deduped | |-- util-deprecate@1.0.2 deduped | +-- pump@3.0.0 | | +-- end-of-stream@1.4.1 deduped | | -- once@1.4.0 deduped | +-- pumpify@1.5.1 | | +-- duplexify@3.6.0 deduped | | +-- inherits@2.0.4 deduped | |-- pump@2.0.1 | | +-- end-of-stream@1.4.1 deduped | | -- once@1.4.0 deduped | +-- stream-each@1.2.2 | | +-- end-of-stream@1.4.1 deduped | |-- stream-shift@1.0.0 deduped | -- through2@2.0.3 | +-- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | |-- safe-buffer@5.1.2 deduped | | -- util-deprecate@1.0.2 deduped |-- xtend@4.0.1 +-- mkdirp@0.5.4 | -- minimist@1.2.5 +-- move-concurrently@1.0.1 | +-- aproba@1.2.0 | +-- copy-concurrently@1.0.5 | | +-- aproba@1.2.0 | | +-- fs-write-stream-atomic@1.0.10 deduped | | +-- iferr@0.1.5 | | +-- mkdirp@0.5.4 deduped | | +-- rimraf@2.7.1 deduped | |-- run-queue@1.0.3 deduped | +-- fs-write-stream-atomic@1.0.10 deduped | +-- mkdirp@0.5.4 deduped | +-- rimraf@2.7.1 deduped | -- run-queue@1.0.3 |-- aproba@1.2.0 +-- node-gyp@5.1.0 | +-- env-paths@2.2.0 | +-- glob@7.1.6 deduped | +-- graceful-fs@4.2.3 deduped | +-- mkdirp@0.5.4 deduped | +-- nopt@4.0.1 deduped | +-- npmlog@4.1.2 deduped | +-- request@2.88.0 deduped | +-- rimraf@2.7.1 deduped | +-- semver@5.7.1 deduped | +-- tar@4.4.13 deduped | -- which@1.3.1 deduped +-- nopt@4.0.1 | +-- abbrev@1.1.1 deduped |-- osenv@0.1.5 deduped +-- normalize-package-data@2.5.0 | +-- hosted-git-info@2.8.8 deduped | +-- resolve@1.10.0 | | -- path-parse@1.0.6 | +-- semver@5.7.1 deduped |-- validate-npm-package-license@3.0.4 deduped +-- npm-audit-report@1.3.2 | +-- cli-table3@0.5.1 deduped | -- console-control-strings@1.1.0 +-- npm-cache-filename@1.0.2 +-- npm-install-checks@3.0.2 |-- semver@5.7.1 deduped +-- npm-lifecycle@3.1.4 | +-- byline@5.0.0 | +-- graceful-fs@4.2.3 deduped | +-- node-gyp@5.1.0 deduped | +-- resolve-from@4.0.0 | +-- slide@1.1.6 deduped | +-- uid-number@0.0.6 deduped | +-- umask@1.1.0 deduped | -- which@1.3.1 deduped +-- npm-package-arg@6.1.1 | +-- hosted-git-info@2.8.8 deduped | +-- osenv@0.1.5 deduped | +-- semver@5.7.1 deduped |-- validate-npm-package-name@3.0.0 deduped +-- npm-packlist@1.4.8 | +-- ignore-walk@3.0.3 | | -- minimatch@3.0.4 deduped | +-- npm-bundled@1.1.1 | |-- npm-normalize-package-bin@1.0.1 deduped | -- npm-normalize-package-bin@1.0.1 deduped +-- npm-pick-manifest@3.0.2 | +-- figgy-pudding@3.5.1 deduped | +-- npm-package-arg@6.1.1 deduped |-- semver@5.7.1 deduped +-- npm-profile@4.0.4 | +-- aproba@2.0.0 deduped | +-- figgy-pudding@3.5.1 deduped | -- npm-registry-fetch@4.0.3 deduped +-- npm-registry-fetch@4.0.3 | +-- bluebird@3.5.5 deduped | +-- figgy-pudding@3.5.1 deduped | +-- JSONStream@1.3.5 deduped | +-- lru-cache@5.1.1 deduped | +-- make-fetch-happen@5.0.2 | | +-- agentkeepalive@3.5.2 | | |-- humanize-ms@1.2.1 | | | -- ms@2.1.1 | | +-- cacache@12.0.3 deduped | | +-- http-cache-semantics@3.8.1 | | +-- http-proxy-agent@2.1.0 | | | +-- agent-base@4.3.0 | | | |-- es6-promisify@5.0.0 | | | | -- es6-promise@4.2.8 | | |-- debug@3.1.0 | | | -- ms@2.0.0 | | +-- https-proxy-agent@2.2.4 | | | +-- agent-base@4.3.0 deduped | | |-- debug@3.1.0 deduped | | +-- lru-cache@5.1.1 deduped | | +-- mississippi@3.0.0 deduped | | +-- node-fetch-npm@2.0.2 | | | +-- encoding@0.1.12 | | | | -- iconv-lite@0.4.23 | | | |-- safer-buffer@2.1.2 deduped | | | +-- json-parse-better-errors@1.0.2 deduped | | | -- safe-buffer@5.1.2 deduped | | +-- promise-retry@1.1.1 deduped | | +-- socks-proxy-agent@4.0.2 | | | +-- agent-base@4.2.1 | | | |-- es6-promisify@5.0.0 deduped | | | -- socks@2.3.3 | | | +-- ip@1.1.5 | | |-- smart-buffer@4.1.0 | | -- ssri@6.0.1 deduped | +-- npm-package-arg@6.1.1 deduped |-- safe-buffer@5.2.0 +-- npm-user-validate@1.0.0 +-- npmlog@4.1.2 | +-- are-we-there-yet@1.1.4 | | +-- delegates@1.0.0 | | -- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | |-- safe-buffer@5.1.2 deduped | | -- util-deprecate@1.0.2 deduped | +-- console-control-strings@1.1.0 deduped | +-- gauge@2.7.4 | | +-- aproba@1.2.0 | | +-- console-control-strings@1.1.0 deduped | | +-- has-unicode@2.0.1 deduped | | +-- object-assign@4.1.1 deduped | | +-- signal-exit@3.0.2 deduped | | +-- string-width@1.0.2 | | | +-- code-point-at@1.1.0 | | | +-- is-fullwidth-code-point@1.0.0 | | | |-- number-is-nan@1.0.1 | | | -- strip-ansi@3.0.1 deduped | | +-- strip-ansi@3.0.1 deduped | |-- wide-align@1.1.2 | | -- string-width@1.0.2 | | +-- code-point-at@1.1.0 deduped | | +-- is-fullwidth-code-point@1.0.0 deduped | |-- strip-ansi@3.0.1 deduped | -- set-blocking@2.0.0 +-- once@1.4.0 |-- wrappy@1.0.2 deduped +-- opener@1.5.1 +-- osenv@0.1.5 | +-- os-homedir@1.0.2 | -- os-tmpdir@1.0.2 +-- pacote@9.5.12 | +-- bluebird@3.5.5 deduped | +-- cacache@12.0.3 deduped | +-- chownr@1.1.4 deduped | +-- figgy-pudding@3.5.1 deduped | +-- get-stream@4.1.0 deduped | +-- glob@7.1.6 deduped | +-- infer-owner@1.0.4 deduped | +-- lru-cache@5.1.1 deduped | +-- make-fetch-happen@5.0.2 deduped | +-- minimatch@3.0.4 deduped | +-- minipass@2.9.0 | | +-- safe-buffer@5.1.2 deduped | |-- yallist@3.0.3 deduped | +-- mississippi@3.0.0 deduped | +-- mkdirp@0.5.4 deduped | +-- normalize-package-data@2.5.0 deduped | +-- npm-normalize-package-bin@1.0.1 deduped | +-- npm-package-arg@6.1.1 deduped | +-- npm-packlist@1.4.8 deduped | +-- npm-pick-manifest@3.0.2 deduped | +-- npm-registry-fetch@4.0.3 deduped | +-- osenv@0.1.5 deduped | +-- promise-inflight@1.0.1 deduped | +-- promise-retry@1.1.1 | | +-- err-code@1.1.2 | | -- retry@0.10.1 | +-- protoduck@5.0.1 | |-- genfun@5.0.0 | +-- rimraf@2.7.1 deduped | +-- safe-buffer@5.1.2 deduped | +-- semver@5.7.1 deduped | +-- ssri@6.0.1 deduped | +-- tar@4.4.13 deduped | +-- unique-filename@1.1.1 deduped | -- which@1.3.1 deduped +-- path-is-inside@1.0.2 +-- promise-inflight@1.0.1 +-- qrcode-terminal@0.12.0 +-- query-string@6.8.2 | +-- decode-uri-component@0.2.0 | +-- split-on-first@1.1.0 |-- strict-uri-encode@2.0.0 +-- qw@1.0.1 +-- read@1.0.7 | -- mute-stream@0.0.7 +-- read-cmd-shim@1.0.5 |-- graceful-fs@4.2.3 deduped +-- read-installed@4.0.3 | +-- debuglog@1.0.1 deduped | +-- graceful-fs@4.2.3 deduped | +-- read-package-json@2.1.1 deduped | +-- readdir-scoped-modules@1.1.0 deduped | +-- semver@5.7.1 deduped | +-- slide@1.1.6 deduped | -- util-extend@1.0.3 +-- read-package-json@2.1.1 | +-- glob@7.1.6 deduped | +-- graceful-fs@4.2.3 deduped | +-- json-parse-better-errors@1.0.2 deduped | +-- normalize-package-data@2.5.0 deduped |-- npm-normalize-package-bin@1.0.1 deduped +-- read-package-tree@5.3.1 | +-- read-package-json@2.1.1 deduped | +-- readdir-scoped-modules@1.1.0 deduped | -- util-promisify@2.1.0 |-- object.getownpropertydescriptors@2.0.3 | +-- define-properties@1.1.3 | | -- object-keys@1.0.12 |-- es-abstract@1.12.0 | +-- es-to-primitive@1.2.0 | | +-- is-callable@1.1.4 deduped | | +-- is-date-object@1.0.1 | | -- is-symbol@1.0.2 | |-- has-symbols@1.0.0 | +-- function-bind@1.1.1 | +-- has@1.0.3 | | -- function-bind@1.1.1 deduped | +-- is-callable@1.1.4 |-- is-regex@1.0.4 | -- has@1.0.3 deduped +-- readable-stream@3.6.0 | +-- inherits@2.0.4 deduped | +-- string_decoder@1.3.0 | |-- safe-buffer@5.2.0 | -- util-deprecate@1.0.2 +-- readdir-scoped-modules@1.1.0 | +-- debuglog@1.0.1 deduped | +-- dezalgo@1.0.3 deduped | +-- graceful-fs@4.2.3 deduped |-- once@1.4.0 deduped +-- request@2.88.0 | +-- aws-sign2@0.7.0 | +-- aws4@1.8.0 | +-- caseless@0.12.0 | +-- combined-stream@1.0.6 | | -- delayed-stream@1.0.0 | +-- extend@3.0.2 | +-- forever-agent@0.6.1 | +-- form-data@2.3.2 | | +-- asynckit@0.4.0 | | +-- combined-stream@1.0.6 deduped | |-- mime-types@2.1.19 deduped | +-- har-validator@5.1.0 | | +-- ajv@5.5.2 | | | +-- co@4.6.0 | | | +-- fast-deep-equal@1.1.0 | | | +-- fast-json-stable-stringify@2.0.0 | | | -- json-schema-traverse@0.3.1 | |-- har-schema@2.0.0 | +-- http-signature@1.2.0 | | +-- assert-plus@1.0.0 | | +-- jsprim@1.4.1 | | | +-- assert-plus@1.0.0 deduped | | | +-- extsprintf@1.3.0 | | | +-- json-schema@0.2.3 | | | -- verror@1.10.0 | | | +-- assert-plus@1.0.0 deduped | | | +-- core-util-is@1.0.2 deduped | | |-- extsprintf@1.3.0 deduped | | -- sshpk@1.14.2 | | +-- asn1@0.2.4 | | |-- safer-buffer@2.1.2 deduped | | +-- assert-plus@1.0.0 deduped | | +-- bcrypt-pbkdf@1.0.2 | | | -- tweetnacl@0.14.5 deduped | | +-- dashdash@1.14.1 | | |-- assert-plus@1.0.0 deduped | | +-- ecc-jsbn@0.1.2 | | | +-- jsbn@0.1.1 deduped | | | -- safer-buffer@2.1.2 deduped | | +-- getpass@0.1.7 | | |-- assert-plus@1.0.0 deduped | | +-- jsbn@0.1.1 | | +-- safer-buffer@2.1.2 | | -- tweetnacl@0.14.5 | +-- is-typedarray@1.0.0 | +-- isstream@0.1.2 | +-- json-stringify-safe@5.0.1 | +-- mime-types@2.1.19 | |-- mime-db@1.35.0 | +-- oauth-sign@0.9.0 | +-- performance-now@2.1.0 | +-- qs@6.5.2 | +-- safe-buffer@5.1.2 deduped | +-- tough-cookie@2.4.3 | | +-- psl@1.1.29 | | -- punycode@1.4.1 | +-- tunnel-agent@0.6.0 | |-- safe-buffer@5.1.2 deduped | -- uuid@3.3.3 deduped +-- retry@0.12.0 +-- rimraf@2.7.1 |-- glob@7.1.6 deduped +-- safe-buffer@5.1.2 +-- semver@5.7.1 +-- sha@3.0.0 | -- graceful-fs@4.2.3 deduped +-- slide@1.1.6 +-- sorted-object@2.0.1 +-- sorted-union-stream@2.1.3 | +-- from2@1.3.0 | | +-- inherits@2.0.4 deduped | |-- readable-stream@1.1.14 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@0.0.1 | | -- string_decoder@0.10.31 |-- stream-iterate@1.2.0 | +-- readable-stream@2.3.6 | | +-- core-util-is@1.0.2 deduped | | +-- inherits@2.0.4 deduped | | +-- isarray@1.0.0 deduped | | +-- process-nextick-args@2.0.0 deduped | | +-- safe-buffer@5.1.2 deduped | | +-- string_decoder@1.1.1 | | | -- safe-buffer@5.1.2 deduped | |-- util-deprecate@1.0.2 deduped | -- stream-shift@1.0.0 deduped +-- ssri@6.0.1 |-- figgy-pudding@3.5.1 deduped +-- stringify-package@1.0.1 +-- tar@4.4.13 | +-- chownr@1.1.4 deduped | +-- fs-minipass@1.2.7 | | -- minipass@2.9.0 | | +-- safe-buffer@5.1.2 deduped | |-- yallist@3.0.3 deduped | +-- minipass@2.9.0 | | +-- safe-buffer@5.1.2 deduped | | -- yallist@3.0.3 deduped | +-- minizlib@1.3.3 | |-- minipass@2.9.0 | | +-- safe-buffer@5.1.2 deduped | | -- yallist@3.0.3 deduped | +-- mkdirp@0.5.4 deduped | +-- safe-buffer@5.1.2 deduped |-- yallist@3.0.3 deduped +-- text-table@0.2.0 +-- tiny-relative-date@1.3.0 +-- uid-number@0.0.6 +-- umask@1.1.0 +-- unique-filename@1.1.1 | -- unique-slug@2.0.0 |-- imurmurhash@0.1.4 deduped +-- unpipe@1.0.0 +-- update-notifier@2.5.0 | +-- boxen@1.3.0 | | +-- ansi-align@2.0.0 | | | -- string-width@2.1.1 deduped | | +-- camelcase@4.1.0 | | +-- chalk@2.4.1 deduped | | +-- cli-boxes@1.0.0 | | +-- string-width@2.1.1 deduped | | +-- term-size@1.2.0 | | |-- execa@0.7.0 | | | +-- cross-spawn@5.1.0 | | | | +-- lru-cache@4.1.5 | | | | | +-- pseudomap@1.0.2 | | | | | -- yallist@2.1.2 | | | | +-- shebang-command@1.2.0 deduped | | | |-- which@1.3.1 deduped | | | +-- get-stream@3.0.0 | | | +-- is-stream@1.1.0 deduped | | | +-- npm-run-path@2.0.2 deduped | | | +-- p-finally@1.0.0 deduped | | | +-- signal-exit@3.0.2 deduped | | | -- strip-eof@1.0.0 deduped | |-- widest-line@2.0.1 | | -- string-width@2.1.1 deduped | +-- chalk@2.4.1 | | +-- ansi-styles@3.2.1 | | |-- color-convert@1.9.1 | | | -- color-name@1.1.3 | | +-- escape-string-regexp@1.0.5 | |-- supports-color@5.4.0 | | -- has-flag@3.0.0 | +-- configstore@3.1.2 | | +-- dot-prop@4.2.0 | | |-- is-obj@1.0.1 | | +-- graceful-fs@4.2.3 deduped | | +-- make-dir@1.3.0 | | | -- pify@3.0.0 | | +-- unique-string@1.0.0 | | |-- crypto-random-string@1.0.0 | | +-- write-file-atomic@2.4.3 deduped | | -- xdg-basedir@3.0.0 deduped | +-- import-lazy@2.1.0 | +-- is-ci@1.2.1 | |-- ci-info@1.6.0 | +-- is-installed-globally@0.1.0 | | +-- global-dirs@0.1.1 | | | -- ini@1.3.5 deduped | |-- is-path-inside@1.0.1 | | -- path-is-inside@1.0.2 deduped | +-- is-npm@1.0.0 | +-- latest-version@3.1.0 | |-- package-json@4.0.1 | | +-- got@6.7.1 | | | +-- create-error-class@3.0.2 | | | | -- capture-stack-trace@1.0.0 | | | +-- duplexer3@0.1.4 | | | +-- get-stream@3.0.0 | | | +-- is-redirect@1.0.0 | | | +-- is-retry-allowed@1.2.0 | | | +-- is-stream@1.1.0 deduped | | | +-- lowercase-keys@1.0.1 | | | +-- safe-buffer@5.1.2 deduped | | | +-- timed-out@4.0.1 | | | +-- unzip-response@2.0.1 | | |-- url-parse-lax@1.0.0 | | | -- prepend-http@1.0.4 | | +-- registry-auth-token@3.4.0 | | | +-- rc@1.2.8 | | | | +-- deep-extend@0.6.0 | | | | +-- ini@1.3.5 deduped | | | | +-- minimist@1.2.5 | | | |-- strip-json-comments@2.0.1 | | | -- safe-buffer@5.1.2 deduped | | +-- registry-url@3.1.0 | | |-- rc@1.2.8 deduped | | -- semver@5.7.1 deduped | +-- semver-diff@2.1.0 | |-- semver@5.7.1 deduped | -- xdg-basedir@3.0.0 +-- uuid@3.3.3 +-- validate-npm-package-license@3.0.4 | +-- spdx-correct@3.0.0 | | +-- spdx-expression-parse@3.0.0 deduped | |-- spdx-license-ids@3.0.3 | -- spdx-expression-parse@3.0.0 | +-- spdx-exceptions@2.1.0 |-- spdx-license-ids@3.0.3 deduped +-- validate-npm-package-name@3.0.0 | -- builtins@1.0.3 +-- which@1.3.1 |-- isexe@2.0.0 +-- worker-farm@1.7.0 | -- errno@0.1.7 |-- prr@1.0.1 -- write-file-atomic@2.4.3 +-- graceful-fs@4.2.3 deduped +-- imurmurhash@0.1.4 deduped -- signal-exit@3.0.2 deduped

INFO[0007] Taking snapshot of full filesystem...
INFO[0007] Resolving paths
INFO[0008] RUN apk upgrade -U
INFO[0008] cmd: /bin/sh
INFO[0008] args: [-c apk upgrade -U]
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz Upgrading critical system libraries and apk-tools: (1/1) Upgrading apk-tools (2.10.4-r3 -> 2.10.5-r0) Executing busybox-1.31.1-r9.trigger Continuing the upgrade transaction with new apk-tools: (1/2) Upgrading libcrypto1.1 (1.1.1d-r3 -> 1.1.1g-r0) (2/2) Upgrading libssl1.1 (1.1.1d-r3 -> 1.1.1g-r0) OK: 7 MiB in 16 packages INFO[0008] Taking snapshot of full filesystem...
INFO[0008] Resolving paths
INFO[0009] RUN npm set registry https://npm.dhe.duke.edu && npm install -g eslint eslint-plugin-html mocha c8 nyc link-checker swagger-tools newman@4 sinon@1 markdown-spellcheck INFO[0009] cmd: /bin/sh
INFO[0009] args: [-c npm set registry https://npm.dhe.duke.edu && npm install -g eslint eslint-plugin-html mocha c8 nyc link-checker swagger-tools newman@4 sinon@1 markdown-spellcheck] npm WARN deprecated mkdirp@0.5.3: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.) npm WARN deprecated formatio@1.1.1: This package is unmaintained. Use @sinonjs/formatio instead npm WARN deprecated samsam@1.1.2: This package has been deprecated in favour of @sinonjs/samsam /usr/local/bin/c8 -> /usr/local/lib/node_modules/c8/bin/c8.js /usr/local/bin/eslint -> /usr/local/lib/node_modules/eslint/bin/eslint.js /usr/local/bin/link-checker -> /usr/local/lib/node_modules/link-checker/cli /usr/local/bin/mdspell -> /usr/local/lib/node_modules/markdown-spellcheck/bin/mdspell /usr/local/bin/mocha -> /usr/local/lib/node_modules/mocha/bin/mocha /usr/local/bin/_mocha -> /usr/local/lib/node_modules/mocha/bin/_mocha /usr/local/bin/newman -> /usr/local/lib/node_modules/newman/bin/newman.js /usr/local/bin/nyc -> /usr/local/lib/node_modules/nyc/bin/nyc.js /usr/local/bin/swagger-tools -> /usr/local/lib/node_modules/swagger-tools/bin/swagger-tools

spawn-sync@1.0.15 postinstall /usr/local/lib/node_modules/markdown-spellcheck/node_modules/spawn-sync node postinstall

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.1 (node_modules/mocha/node_modules/chokidar/node_modules/fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code EUIDLOOKUP npm ERR! lifecycle could not get uid/gid npm ERR! lifecycle [ 'nobody', 0 ] npm ERR! lifecycle npm ERR! npm ERR! Failed to look up the user/group for running scripts. npm ERR! npm ERR! Try again with a different --user or --group settings, or npm ERR! run with --unsafe-perm to execute scripts as root.

npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2020-04-23T11_31_03_219Z-debug.log error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1 Running after_script Uploading artifacts for failed job ERROR: Job failed: command terminated with exit code 1

[tejal29]

@jeunii Can you verify if the following capabilities are set for the kaniko pod?

https://github.com/GoogleContainerTools/kaniko/issues/1020#issuecomment-583565813

[ddgenome]

This issue is likely with the version of COS_CONTAINERD you are using. As mentioned here, https://github.com/containerd/containerd/issues/4182#issuecomment-646798361 , the issue has been confirmed by GCP support.

Google Cloud support has informed me this is an issue with COS, not containerd. COS only allows paths under /var/lib/docker to be mounted with the exec option. Since containerd layers are mounted from /var/lib/containerd, they are mounted with the noexec option. So it seems like the fix needs to be made in COS, adding /var/lib/containerd to the list of paths that can be mounted with the exec option.