search

GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.94k stars 1.44k forks source link

Installed binaries are missing from image #2049

Closed devent closed 2 years ago

devent commented 2 years ago

Actual behavior

Binaries are missing from /usr/bin

Expected behavior

ssh-agent, git, gpg, etc. are all present in /usr/bin

To Reproduce

  1. Build the docker file
  2. Push to Harbor
  3. Pull locally
  4. Run the image
  5. Run ssh, git or gpg

Additional Information

FROM debian:bullseye-20220228
LABEL maintainer "Erwin Müller <erwin@muellerpublic.de>"

USER root

ENV DEBUG="false"
ENV DEBIAN_FRONTEND=noninteractive
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000

RUN groupadd -g ${gid} ${group}
RUN useradd -c "Jenkins user" -d /home/${user} -u ${uid} -g ${gid} -m ${user}
LABEL Description="Image with Helm installed"

ARG AGENT_WORKDIR=/home/${user}/agent

ENV LANG C.UTF-8

ADD rootfs/ /

RUN set -ex; \
    apt-get update; \
    apt-get upgrade -y; \
    apt-get install -y curl openssh-client gnupg gpg-agent git make; \
    ls -al /usr/bin; \
    rm -rf /var/lib/apt/lists/*

ARG HELM_VERSION=3.8.1
ARG USER_HOME_DIR="/root"
ARG SHA=d643f48fe28eeb47ff68a1a7a26fc5142f348d02c8bc38d699674016716f61cd
ARG BASE_URL=https://get.helm.sh

RUN set -ex; \
    mkdir -p /usr/share/helm; \
    curl -fsSL -o /tmp/helm.tar.gz ${BASE_URL}/helm-v${HELM_VERSION}-linux-amd64.tar.gz; \
    echo "${SHA} /tmp/helm.tar.gz" | sha256sum -c - ; \
    tar -xzf /tmp/helm.tar.gz -C /usr/share/helm --strip-components=1; \
    rm -f /tmp/helm.tar.gz

RUN set -ex; \
    apt-get remove -y curl libcurl4; \
    ls -al /usr/bin

ENV HELM_HOME /usr/share/helm

ENV PATH ${HELM_HOME}:${PATH}

RUN set -ex; \
    mkdir -p /home/${user}/.jenkins; \
    mkdir -p ${AGENT_WORKDIR}; \
    chown ${user}.${group} -R /home/${user} ${AGENT_WORKDIR}

USER ${user}

ENV PATH=${PATH}

ENV AGENT_WORKDIR=${AGENT_WORKDIR}

VOLUME /home/${user}/.jenkins
VOLUME ${AGENT_WORKDIR}
WORKDIR /home/${user}

RUN set -ex; \
    ls -al /usr/bin

ENTRYPOINT ["/docker-entrypoint.sh"]

RUN set -ex; \
    ls -al /usr/bin
Description Yes/No
Please check if this a new feature you are proposing
  • - [ no ]
Please check if the build works in docker but not in kaniko
  • - [ yes ]
Please check if this error is seen when you use --cache flag
  • - [ yes ]
Please check if your dockerfile is a multistage dockerfile
  • - [ no ]

Here is the log output that clearly shows the binaries installed on the last layer:

[36mINFO[0025] Running: [/bin/sh -c set -ex;     ls -al /usr/bin] 
+ ls -al /usr/bin
total 43152
drwxr-xr-x  2 root root      12288 Apr 13 07:12 .
drwxr-xr-x 11 root root       4096 Apr 13 06:19 ..
-rwxr-xr-x  1 root root      60224 Sep 24  2020 [
-rwxr-xr-x  1 root root      26856 Jan 20 20:10 addpart
-rwxr-xr-x  1 root root      18664 Jun 10  2021 apt
-rwxr-xr-x  1 root root      88376 Jun 10  2021 apt-cache
-rwxr-xr-x  1 root root      26936 Jun 10  2021 apt-cdrom
-rwxr-xr-x  1 root root      26856 Jun 10  2021 apt-config
-rwxr-xr-x  1 root root      47416 Jun 10  2021 apt-get
-rwxr-xr-x  1 root root      28191 Jun 10  2021 apt-key
-rwxr-xr-x  1 root root      51512 Jun 10  2021 apt-mark
-rwxr-xr-x  1 root root      39744 Sep 24  2020 arch
lrwxrwxrwx  1 root root         21 Apr 13 07:12 awk -> /etc/alternatives/awk
-rwxr-xr-x  1 root root      60352 Sep 24  2020 b2sum
-rwxr-xr-x  1 root root      43872 Sep 24  2020 base32
-rwxr-xr-x  1 root root      43872 Sep 24  2020 base64
-rwxr-xr-x  1 root root      39712 Sep 24  2020 basename
-rwxr-xr-x  1 root root      56160 Sep 24  2020 basenc
-rwxr-xr-x  1 root root       6759 Aug  4  2021 bashbug
-rwxr-xr-x  1 root root       6332 Mar 18 18:25 c_rehash
lrwxrwxrwx  1 root root          3 Apr 13 07:12 captoinfo -> tic
-rwxr-xr-x  1 root root       3304 Mar 17 21:37 catchsegv
-rwxr-sr-x  1 root shadow    80256 Feb  7  2020 chage
-rwxr-xr-x  1 root root      14496 Jun  7  2021 chattr
-rwxr-xr-x  1 root root      72768 Sep 24  2020 chcon
-rwsr-xr-x  1 root root      58416 Feb  7  2020 chfn
-rwxr-xr-x  1 root root      51432 Jan 20 20:10 choom
-rwxr-xr-x  1 root root      35048 Jan 20 20:10 chrt
-rwsr-xr-x  1 root root      52880 Feb  7  2020 chsh
-rwxr-xr-x  1 root root      39712 Sep 24  2020 cksum
-rwxr-xr-x  1 root root      14496 Jan  1  2021 clear
-rwxr-xr-x  1 root root      14648 Aug  4  2021 clear_console
-rwxr-xr-x  1 root root      47904 Jan  1  2021 cmp
-rwxr-xr-x  1 root root      43872 Sep 24  2020 comm
-rwxr-xr-x  1 root root      15372 Sep 24  2021 corelist
-rwxr-xr-x  1 root root       8357 Sep 24  2021 cpan
-rwxr-xr-x  1 root root       8378 Sep 24  2021 cpan5.32-x86_64-linux-gnu
-rwxr-xr-x  1 root root     121920 Sep 24  2020 csplit
-rwxr-xr-x  1 root root      48032 Sep 24  2020 cut
-rwxr-xr-x  1 root root      21394 Dec 14  2020 deb-systemd-helper
-rwxr-xr-x  1 root root       4411 Dec 14  2020 deb-systemd-invoke
-rwxr-xr-x  1 root root       2859 Jun 10  2021 debconf
-rwxr-xr-x  1 root root      11541 Jun 10  2021 debconf-apt-progress
-rwxr-xr-x  1 root root        608 Jun 10  2021 debconf-communicate
-rwxr-xr-x  1 root root       1719 Jun 10  2021 debconf-copydb
-rwxr-xr-x  1 root root        647 Jun 10  2021 debconf-escape
-rwxr-xr-x  1 root root       2935 Jun 10  2021 debconf-set-selections
-rwxr-xr-x  1 root root       1827 Jun 10  2021 debconf-show
-rwxr-xr-x  1 root root      26856 Jan 20 20:10 delpart
-rwxr-xr-x  1 root root     208320 Jan  1  2021 diff
-rwxr-xr-x  1 root root      64512 Jan  1  2021 diff3
-rwxr-xr-x  1 root root      47944 Sep 24  2020 dircolors
-rwxr-xr-x  1 root root     579216 Jan 27 19:46 dirmngr
-rwxr-xr-x  1 root root     109344 Jan 27 19:46 dirmngr-client
-rwxr-xr-x  1 root root      39712 Sep 24  2020 dirname
-rwxr-xr-x  1 root root     313888 Apr 13  2021 dpkg
-rwxr-xr-x  1 root root     162184 Apr 13  2021 dpkg-deb
-rwxr-xr-x  1 root root     154048 Apr 13  2021 dpkg-divert
-rwxr-xr-x  1 root root      21162 Apr 13  2021 dpkg-maintscript-helper
-rwxr-xr-x  1 root root     166328 Apr 13  2021 dpkg-query
-rwxr-xr-x  1 root root       4149 Apr 13  2021 dpkg-realpath
-rwxr-xr-x  1 root root     125272 Apr 13  2021 dpkg-split
-rwxr-xr-x  1 root root      63608 Apr 13  2021 dpkg-statoverride
-rwxr-xr-x  1 root root      80216 Apr 13  2021 dpkg-trigger
-rwxr-xr-x  1 root root     175360 Sep 24  2020 du
-rwxr-xr-x  1 root root      41816 Sep 24  2021 enc2xs
-rwxr-xr-x  1 root root       3066 Sep 24  2021 encguess
-rwxr-xr-x  1 root root      48480 Sep 24  2020 env
-rwxr-xr-x  1 root root      43872 Sep 24  2020 expand
-rwxr-sr-x  1 root shadow    31160 Feb  7  2020 expiry
-rwxr-xr-x  1 root root     117728 Sep 24  2020 expr
-rwxr-xr-x  1 root root      81056 Sep 24  2020 factor
-rwxr-xr-x  1 root root      22984 Feb  7  2020 faillog
-rwxr-xr-x  1 root root      35048 Jan 20 20:10 fallocate
-rwxr-xr-x  1 root root      18952 May 15  2020 fc-cache
-rwxr-xr-x  1 root root      14800 May 15  2020 fc-cat
-rwxr-xr-x  1 root root      14616 May 15  2020 fc-conflist
-rwxr-xr-x  1 root root      14680 May 15  2020 fc-list
-rwxr-xr-x  1 root root      14744 May 15  2020 fc-match
-rwxr-xr-x  1 root root      14672 May 15  2020 fc-pattern
-rwxr-xr-x  1 root root      14640 May 15  2020 fc-query
-rwxr-xr-x  1 root root      14688 May 15  2020 fc-scan
-rwxr-xr-x  1 root root      14688 May 15  2020 fc-validate
-rwxr-xr-x  1 root root      35096 Jan 20 20:10 fincore
-rwxr-xr-x  1 root root     311008 Jan  9  2021 find
-rwxr-xr-x  1 root root      35128 Jan 20 20:10 flock
-rwxr-xr-x  1 root root      47936 Sep 24  2020 fmt
-rwxr-xr-x  1 root root      43840 Sep 24  2020 fold
-rwxr-xr-x  1 root root      35192 Mar 17 21:37 getconf
-rwxr-xr-x  1 root root      32096 Mar 17 21:37 getent
-rwxr-xr-x  1 root root      22760 Jan 20 20:10 getopt
-rwxr-xr-x  1 root root    3343280 Mar 10  2021 git
-rwxr-xr-x  1 root root   11052952 Jul 18  2021 git-lfs
lrwxrwxrwx  1 root root          3 Mar 10  2021 git-receive-pack -> git
-rwxr-xr-x  1 root root    1733608 Mar 10  2021 git-shell
lrwxrwxrwx  1 root root          3 Mar 10  2021 git-upload-archive -> git
lrwxrwxrwx  1 root root          3 Mar 10  2021 git-upload-pack -> git
lrwxrwxrwx  1 root root          4 Apr 10  2021 gmake -> make
-rwsr-xr-x  1 root root      88304 Feb  7  2020 gpasswd
-rwxr-xr-x  1 root root    1099632 Jan 27 19:46 gpg
-rwxr-xr-x  1 root root     410440 Jan 27 19:46 gpg-agent
-rwxr-xr-x  1 root root     150400 Jan 27 19:46 gpg-connect-agent
-rwxr-xr-x  1 root root     195504 Jan 27 19:46 gpg-wks-server
-rwxr-xr-x  1 root root       3516 Jan 27 19:46 gpg-zip
-rwxr-xr-x  1 root root     915600 Jan 27 19:46 gpgcompose
-rwxr-xr-x  1 root root     166496 Jan 27 19:46 gpgconf
-rwxr-xr-x  1 root root      30952 Jan 27 19:46 gpgparsemail
-rwxr-xr-x  1 root root     510976 Jan 27 19:46 gpgsm
-rwxr-xr-x  1 root root      72176 Jan 27 19:46 gpgsplit
-rwxr-xr-x  1 root root     146688 Jan 27 19:46 gpgtar
-rwxr-xr-x  1 root root     449400 Jan 27 19:46 gpgv
-rwxr-xr-x  1 root root      43840 Sep 24  2020 groups
-rwxr-xr-x  1 root root      29224 Sep 24  2021 h2ph
-rwxr-xr-x  1 root root      60931 Sep 24  2021 h2xs
-rwxr-xr-x  1 root root      48000 Sep 24  2020 head
-rwxr-xr-x  1 root root      39680 Sep 24  2020 hostid
lrwxrwxrwx  1 root root          7 Apr 13 07:12 i386 -> setarch
-rwxr-xr-x  1 root root      60424 Mar 17 21:37 iconv
-rwxr-xr-x  1 root root      48064 Sep 24  2020 id
-rwxr-xr-x  1 root root      63720 Jan  1  2021 infocmp
lrwxrwxrwx  1 root root          3 Apr 13 07:12 infotocap -> tic
-rwxr-xr-x  1 root root     159528 Sep 24  2020 install
-rwxr-xr-x  1 root root       4370 Sep 24  2021 instmodsh
-rwxr-xr-x  1 root root      30952 Jan 20 20:10 ionice
-rwxr-xr-x  1 root root      31024 Jan 20 20:10 ipcmk
-rwxr-xr-x  1 root root      35048 Jan 20 20:10 ipcrm
-rwxr-xr-x  1 root root      71912 Jan 20 20:10 ipcs
-rwxr-xr-x  1 root root      14608 Sep 27  2020 ischroot
-rwxr-xr-x  1 root root      56224 Sep 24  2020 join
-rwxr-xr-x  1 root root       4989 Sep 24  2021 json_pp
-rwxr-xr-x  1 root root     162504 Jan 27 19:46 kbxutil
-rwxr-xr-x  1 root root      47336 Jan 20 20:10 last
lrwxrwxrwx  1 root root          4 Apr 13 07:12 lastb -> last
-rwxr-xr-x  1 root root      32256 Feb  7  2020 lastlog
-rwxr-xr-x  1 root root       7784 Jun 16  2020 lcf
-rwxr-xr-x  1 root root       5401 Mar 17 21:37 ldd
-rwxr-xr-x  1 root root     175816 Jul  5  2020 less
-rwxr-xr-x  1 root root      14504 Jul  5  2020 lessecho
lrwxrwxrwx  1 root root          8 Jul  5  2020 lessfile -> lesspipe
-rwxr-xr-x  1 root root      24136 Jul  5  2020 lesskey
-rwxr-xr-x  1 root root       8543 Jul  5  2020 lesspipe
-rwxr-xr-x  1 root root      15775 Sep 24  2021 libnetcfg
-rwxr-xr-x  1 root root      39680 Sep 24  2020 link
lrwxrwxrwx  1 root root          7 Apr 13 07:12 linux32 -> setarch
lrwxrwxrwx  1 root root          7 Apr 13 07:12 linux64 -> setarch
-rwxr-xr-x  1 root root      55336 Mar 17 21:37 locale
-rwxr-xr-x  1 root root     315136 Mar 17 21:37 localedef
-rwxr-xr-x  1 root root      52040 Jan 20 20:10 logger
-rwxr-xr-x  1 root root      39712 Sep 24  2020 logname
-rwxr-xr-x  1 root root      14496 Jun  7  2021 lsattr
-rwxr-xr-x  1 root root     100584 Jan 20 20:10 lscpu
-rwxr-xr-x  1 root root      96488 Jan 20 20:10 lsipc
-rwxr-xr-x  1 root root      39480 Jan 20 20:10 lslocks
-rwxr-xr-x  1 root root      67816 Jan 20 20:10 lslogins
-rwxr-xr-x  1 root root      67816 Jan 20 20:10 lsmem
-rwxr-xr-x  1 root root      51440 Jan 20 20:10 lsns
-rwxr-xr-x  1 root root       1081 Aug 28  2017 lspgpot
-rwxr-xr-x  1 root root     240280 Apr 10  2021 make
-rwxr-xr-x  1 root root       4905 Apr 10  2021 make-first-existing-target
-rwxr-xr-x  1 root root     158296 Feb 16  2020 mawk
-rwxr-xr-x  1 root root      35120 Jan 20 20:10 mcookie
-rwxr-xr-x  1 root root      48032 Sep 24  2020 md5sum
lrwxrwxrwx  1 root root          6 Apr 13 07:12 md5sum.textutils -> md5sum
-rwxr-xr-x  1 root root      14568 Jan 20 20:10 mesg
-rwxr-xr-x  1 root root       3060 Jan  8 18:26 migrate-pubring-from-classic-gpg
-rwxr-xr-x  1 root root      68704 Sep 24  2020 mkfifo
-rwxr-xr-x  1 root root      35048 Jan 20 20:10 namei
lrwxrwxrwx  1 root root         22 Apr 13 07:12 nawk -> /etc/alternatives/nawk
-rwsr-xr-x  1 root root      44632 Feb  7  2020 newgrp
-rwxr-xr-x  1 root root      43840 Sep 24  2020 nice
-rwxr-xr-x  1 root root     113696 Sep 24  2020 nl
-rwxr-xr-x  1 root root      43872 Sep 24  2020 nohup
-rwxr-xr-x  1 root root      43840 Sep 24  2020 nproc
-rwxr-xr-x  1 root root      35280 Jan 20 20:10 nsenter
-rwxr-xr-x  1 root root      68576 Sep 24  2020 numfmt
-rwxr-xr-x  1 root root      72640 Sep 24  2020 od
-rwxr-xr-x  1 root root     736792 Mar 18 18:25 openssl
lrwxrwxrwx  1 root root         23 Apr 13 07:12 pager -> /etc/alternatives/pager
-rwxr-xr-x  1 root root     121072 Jan 20 20:10 partx
-rwsr-xr-x  1 root root      63960 Feb  7  2020 passwd
-rwxr-xr-x  1 root root      43872 Sep 24  2020 paste
-rwxr-xr-x  1 root root     191936 Jan  9  2021 patch
-rwxr-xr-x  1 root root      39712 Sep 24  2020 pathchk
-rwxr-xr-x  2 root root    3681152 Sep 24  2021 perl
-rwxr-xr-x  1 root root      14688 Sep 24  2021 perl5.32-x86_64-linux-gnu
-rwxr-xr-x  2 root root    3681152 Sep 24  2021 perl5.32.1
-rwxr-xr-x  2 root root      45262 Sep 24  2021 perlbug
-rwxr-xr-x  1 root root        125 Aug 25  2021 perldoc
-rwxr-xr-x  1 root root      10864 Sep 24  2021 perlivp
-rwxr-xr-x  2 root root      45262 Sep 24  2021 perlthanks
-rwxr-xr-x  1 root root       8357 Sep 24  2021 piconv
lrwxrwxrwx  1 root root         26 Apr 13 06:19 pinentry -> /etc/alternatives/pinentry
-rwxr-xr-x  1 root root      63992 May  8  2020 pinentry-curses
-rwxr-xr-x  1 root root      44000 Sep 24  2020 pinky
-rwxr-xr-x  1 root root       4533 Sep 24  2021 pl2pm
-rwxr-xr-x  1 root root      23112 Mar 17 21:37 pldd
-rwxr-xr-x  1 root root       4134 Sep 24  2021 pod2html
-rwxr-xr-x  1 root root      15034 Sep 24  2021 pod2man
-rwxr-xr-x  1 root root      10803 Sep 24  2021 pod2text
-rwxr-xr-x  1 root root       3948 Sep 24  2021 pod2usage
-rwxr-xr-x  1 root root       3658 Sep 24  2021 podchecker
-rwxr-xr-x  1 root root      76896 Sep 24  2020 pr
-rwxr-xr-x  1 root root      39680 Sep 24  2020 printenv
-rwxr-xr-x  1 root root      60256 Sep 24  2020 printf
-rwxr-xr-x  1 root root      39672 Jan 20 20:10 prlimit
-rwxr-xr-x  1 root root      13655 Sep 24  2021 prove
-rwxr-xr-x  1 root root       3549 Sep 24  2021 ptar
-rwxr-xr-x  1 root root       2628 Sep 24  2021 ptardiff
-rwxr-xr-x  1 root root       4392 Sep 24  2021 ptargrep
-rwxr-xr-x  1 root root     138336 Sep 24  2020 ptx
lrwxrwxrwx  1 root root         21 Apr 13 06:41 rcp -> /etc/alternatives/rcp
-rwxr-xr-x  1 root root      56160 Sep 24  2020 realpath
-rwxr-xr-x  1 root root      14560 Jan 20 20:10 renice
lrwxrwxrwx  1 root root          4 Apr 13 07:12 reset -> tset
-rwxr-xr-x  1 root root      63720 Jan 20 20:10 resizepart
-rwxr-xr-x  1 root root      14568 Jan 20 20:10 rev
-rwxr-xr-x  1 root root         30 Jan 29  2020 rgrep
lrwxrwxrwx  1 root root         24 Apr 13 06:41 rlogin -> /etc/alternatives/rlogin
lrwxrwxrwx  1 root root         21 Apr 13 06:41 rsh -> /etc/alternatives/rsh
-rwxr-xr-x  1 root root      43936 Sep 24  2020 runcon
-rwxr-xr-x  1 root root      10481 Sep 27  2020 savelog
-rwxr-xr-x  1 root root     116880 Mar 13  2021 scp
-rwxr-xr-x  1 root root      67816 Jan 20 20:10 script
-rwxr-xr-x  1 root root      55528 Jan 20 20:10 scriptlive
-rwxr-xr-x  1 root root      43240 Jan 20 20:10 scriptreplay
-rwxr-xr-x  1 root root      48032 Jan  1  2021 sdiff
-rwxr-xr-x  1 root root       2446 Jan 12  2021 select-editor
-rwxr-xr-x  1 root root       1230 Jan 12  2021 sensible-browser
-rwxr-xr-x  1 root root       1178 Jan 12  2021 sensible-editor
-rwxr-xr-x  1 root root        452 Jan 12  2021 sensible-pager
-rwxr-xr-x  1 root root      56192 Sep 24  2020 seq
-rwxr-xr-x  1 root root      27136 Jan 20 20:10 setarch
-rwxr-xr-x  1 root root      51440 Jan 20 20:10 setpriv
-rwxr-xr-x  1 root root      14568 Jan 20 20:10 setsid
-rwxr-xr-x  1 root root      47336 Jan 20 20:10 setterm
-rwxr-xr-x  1 root root     174440 Mar 13  2021 sftp
lrwxrwxrwx  1 root root          6 Apr 13 07:12 sg -> newgrp
-rwxr-xr-x  1 root root      52128 Sep 24  2020 sha1sum
-rwxr-xr-x  1 root root      64416 Sep 24  2020 sha224sum
-rwxr-xr-x  1 root root      64416 Sep 24  2020 sha256sum
-rwxr-xr-x  1 root root      68512 Sep 24  2020 sha384sum
-rwxr-xr-x  1 root root      68512 Sep 24  2020 sha512sum
-rwxr-xr-x  1 root root       9976 Sep 24  2021 shasum
-rwxr-xr-x  1 root root      64608 Sep 24  2020 shred
-rwxr-xr-x  1 root root      60384 Sep 24  2020 shuf
lrwxrwxrwx  1 root root          3 Mar 13  2021 slogin -> ssh
-rwxr-xr-x  1 root root     118408 Sep 24  2020 sort
-rwxr-xr-x  1 root root      19411 Sep 24  2021 splain
-rwxr-xr-x  1 root root      60896 Sep 24  2020 split
-rwxr-xr-x  1 root root     797480 Mar 13  2021 ssh
-rwxr-xr-x  1 root root     374912 Mar 13  2021 ssh-add
-rwxr-sr-x  1 root ssh      354440 Mar 13  2021 ssh-agent
-rwxr-xr-x  1 root root       1455 Mar 13  2021 ssh-argv0
-rwxr-xr-x  1 root root      10950 Mar 13  2021 ssh-copy-id
-rwxr-xr-x  1 root root     485576 Mar 13  2021 ssh-keygen
-rwxr-xr-x  1 root root     469224 Mar 13  2021 ssh-keyscan
-rwxr-xr-x  1 root root      85184 Sep 24  2020 stat
-rwxr-xr-x  1 root root      52064 Sep 24  2020 stdbuf
-rwxr-xr-x  1 root root       5521 Sep 24  2021 streamzip
-rwxr-xr-x  1 root root      48008 Sep 24  2020 sum
-rwxr-xr-x  1 root root      18584 Jan  1  2021 tabs
-rwxr-xr-x  1 root root     109536 Sep 24  2020 tac
-rwxr-xr-x  1 root root      76864 Sep 24  2020 tail
-rwxr-xr-x  1 root root      35048 Jan 20 20:10 taskset
-rwxr-xr-x  1 root root      43904 Sep 24  2020 tee
-rwxr-xr-x  1 root root      56128 Sep 24  2020 test
-rwxr-xr-x  1 root root      92432 Jan  1  2021 tic
-rwxr-xr-x  1 root root      44448 Sep 24  2020 timeout
-rwxr-xr-x  1 root root      22680 Jan  1  2021 toe
lrwxrwxrwx  1 root root         10 Apr 13 07:12 touch -> /bin/touch
-rwxr-xr-x  1 root root      26808 Jan  1  2021 tput
-rwxr-xr-x  1 root root      56160 Sep 24  2020 tr
-rwxr-xr-x  1 root root      43840 Sep 24  2020 truncate
-rwxr-xr-x  1 root root      30880 Jan  1  2021 tset
-rwxr-xr-x  1 root root      56128 Sep 24  2020 tsort
-rwxr-xr-x  1 root root      39712 Sep 24  2020 tty
-rwxr-xr-x  1 root root      15352 Mar 17 21:37 tzselect
-rwxr-xr-x  1 root root      41883 Jun 16  2020 ucf
-rwxr-xr-x  1 root root      19367 Jun 16  2020 ucfq
-rwxr-xr-x  1 root root      10719 Jun 16  2020 ucfr
-rwxr-xr-x  1 root root      43872 Sep 24  2020 unexpand
-rwxr-xr-x  1 root root      52128 Sep 24  2020 uniq
-rwxr-xr-x  1 root root      39680 Sep 24  2020 unlink
-rwxr-xr-x  1 root root      47576 Jan 20 20:10 unshare
-rwxr-xr-x  1 root root      59480 Apr 13  2021 update-alternatives
-rwxr-xr-x  1 root root      39744 Sep 24  2020 users
-rwxr-xr-x  1 root root      30952 Jan 20 20:10 utmpdump
-rwxr-sr-x  1 root tty       35048 Jan 20 20:10 wall
-rwxr-xr-x  1 root root      18584 Jan 27 19:46 watchgnupg
-rwxr-xr-x  1 root root      48072 Sep 24  2020 wc
-rwxr-xr-x  1 root root      31400 Jan 20 20:10 whereis
-rwxr-xr-x  1 root root        946 Sep 27  2020 which
-rwxr-xr-x  1 root root      60384 Sep 24  2020 who
-rwxr-xr-x  1 root root      39712 Sep 24  2020 whoami
lrwxrwxrwx  1 root root          7 Apr 13 07:12 x86_64 -> setarch
-rwxr-xr-x  1 root root      76152 Jan  9  2021 xargs
-rwxr-xr-x  1 root root      52680 Jan  5  2021 xauth
-rwxr-xr-x  1 root root       5164 Sep 24  2021 xsubpp
-rwxr-xr-x  1 root root      39680 Sep 24  2020 yes
-rwxr-xr-x  1 root root      22936 Mar 17 21:37 zdump
-rwxr-xr-x  1 root root      51239 Sep 24  2021 zipdetails
INFO[0025] Taking snapshot of full filesystem...        
INFO[0026] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0026] Pushing image to harbor.anrisoftware.com/robobeerun/jenkins-helm:3.8.1-1.0.0-snapshot 
INFO[0029] Pushed harbor.anrisoftware.com/robobeerun/jenkins-helm@sha256:34de3dc721b3b9780858b0135e29303920aed519a185ac11a9abfcd9fb32fabc

But if I try to run the image the binaries are no longer there:

jenkins@736e01257b04:~$ ls -al /usr/bin
total 18040
drwxr-xr-x 1 root root      4096 Apr 13 07:12  .
drwxr-xr-x 1 root root      4096 Apr 13 06:19  ..
-rwxr-xr-x 1 root root     60224 Sep 24  2020 '['
-rwxr-xr-x 1 root root     26856 Jan 20 20:10  addpart
-rwxr-xr-x 1 root root     18664 Jun 10  2021  apt
-rwxr-xr-x 1 root root     88376 Jun 10  2021  apt-cache
-rwxr-xr-x 1 root root     26936 Jun 10  2021  apt-cdrom
-rwxr-xr-x 1 root root     26856 Jun 10  2021  apt-config
-rwxr-xr-x 1 root root     47416 Jun 10  2021  apt-get
-rwxr-xr-x 1 root root     28191 Jun 10  2021  apt-key
-rwxr-xr-x 1 root root     51512 Jun 10  2021  apt-mark
-rwxr-xr-x 1 root root     39744 Sep 24  2020  arch
lrwxrwxrwx 1 root root        21 Feb 28 00:00  awk -> /etc/alternatives/awk
-rwxr-xr-x 1 root root     60352 Sep 24  2020  b2sum
-rwxr-xr-x 1 root root     43872 Sep 24  2020  base32
-rwxr-xr-x 1 root root     43872 Sep 24  2020  base64
-rwxr-xr-x 1 root root     39712 Sep 24  2020  basename
-rwxr-xr-x 1 root root     56160 Sep 24  2020  basenc
-rwxr-xr-x 1 root root      6759 Aug  4  2021  bashbug
lrwxrwxrwx 1 root root         3 Jan  1  2021  captoinfo -> tic
-rwxr-xr-x 1 root root      3304 Mar 17 21:37  catchsegv
-rwxr-sr-x 1 root shadow   80256 Feb  7  2020  chage
-rwxr-xr-x 1 root root     14496 Jun  7  2021  chattr
-rwxr-xr-x 1 root root     72768 Sep 24  2020  chcon
-rwsr-xr-x 1 root root     58416 Feb  7  2020  chfn
-rwxr-xr-x 1 root root     51432 Jan 20 20:10  choom
-rwxr-xr-x 1 root root     35048 Jan 20 20:10  chrt
-rwsr-xr-x 1 root root     52880 Feb  7  2020  chsh
-rwxr-xr-x 1 root root     39712 Sep 24  2020  cksum
-rwxr-xr-x 1 root root     14496 Jan  1  2021  clear
-rwxr-xr-x 1 root root     14648 Aug  4  2021  clear_console
-rwxr-xr-x 1 root root     47904 Jan  1  2021  cmp
-rwxr-xr-x 1 root root     43872 Sep 24  2020  comm
-rwxr-xr-x 1 root root    121920 Sep 24  2020  csplit
-rwxr-xr-x 1 root root     48032 Sep 24  2020  cut
-rwxr-xr-x 1 root root     21394 Dec 14  2020  deb-systemd-helper
-rwxr-xr-x 1 root root      4411 Dec 14  2020  deb-systemd-invoke
-rwxr-xr-x 1 root root      2859 Jun 10  2021  debconf
-rwxr-xr-x 1 root root     11541 Jun 10  2021  debconf-apt-progress
-rwxr-xr-x 1 root root       608 Jun 10  2021  debconf-communicate
-rwxr-xr-x 1 root root      1719 Jun 10  2021  debconf-copydb
-rwxr-xr-x 1 root root       647 Jun 10  2021  debconf-escape
-rwxr-xr-x 1 root root      2935 Jun 10  2021  debconf-set-selections
-rwxr-xr-x 1 root root      1827 Jun 10  2021  debconf-show
-rwxr-xr-x 1 root root     26856 Jan 20 20:10  delpart
-rwxr-xr-x 1 root root    208320 Jan  1  2021  diff
-rwxr-xr-x 1 root root     64512 Jan  1  2021  diff3
-rwxr-xr-x 1 root root     47944 Sep 24  2020  dircolors
-rwxr-xr-x 1 root root     39712 Sep 24  2020  dirname
-rwxr-xr-x 1 root root    313888 Apr 13  2021  dpkg
-rwxr-xr-x 1 root root    162184 Apr 13  2021  dpkg-deb
-rwxr-xr-x 1 root root    154048 Apr 13  2021  dpkg-divert
-rwxr-xr-x 1 root root     21162 Apr 13  2021  dpkg-maintscript-helper
-rwxr-xr-x 1 root root    166328 Apr 13  2021  dpkg-query
-rwxr-xr-x 1 root root      4149 Apr 13  2021  dpkg-realpath
-rwxr-xr-x 1 root root    125272 Apr 13  2021  dpkg-split
-rwxr-xr-x 1 root root     63608 Apr 13  2021  dpkg-statoverride
-rwxr-xr-x 1 root root     80216 Apr 13  2021  dpkg-trigger
-rwxr-xr-x 1 root root    175360 Sep 24  2020  du
-rwxr-xr-x 1 root root     48480 Sep 24  2020  env
-rwxr-xr-x 1 root root     43872 Sep 24  2020  expand
-rwxr-sr-x 1 root shadow   31160 Feb  7  2020  expiry
-rwxr-xr-x 1 root root    117728 Sep 24  2020  expr
-rwxr-xr-x 1 root root     81056 Sep 24  2020  factor
-rwxr-xr-x 1 root root     22984 Feb  7  2020  faillog
-rwxr-xr-x 1 root root     35048 Jan 20 20:10  fallocate
-rwxr-xr-x 1 root root     35096 Jan 20 20:10  fincore
-rwxr-xr-x 1 root root    311008 Jan  9  2021  find
-rwxr-xr-x 1 root root     35128 Jan 20 20:10  flock
-rwxr-xr-x 1 root root     47936 Sep 24  2020  fmt
-rwxr-xr-x 1 root root     43840 Sep 24  2020  fold
-rwxr-xr-x 1 root root     35192 Mar 17 21:37  getconf
-rwxr-xr-x 1 root root     32096 Mar 17 21:37  getent
-rwxr-xr-x 1 root root     22760 Jan 20 20:10  getopt
-rwsr-xr-x 1 root root     88304 Feb  7  2020  gpasswd
-rwxr-xr-x 1 root root    449400 Jan 27 19:46  gpgv
-rwxr-xr-x 1 root root     43840 Sep 24  2020  groups
-rwxr-xr-x 1 root root     48000 Sep 24  2020  head
-rwxr-xr-x 1 root root     39680 Sep 24  2020  hostid
lrwxrwxrwx 1 root root         7 Jan 20 20:10  i386 -> setarch
-rwxr-xr-x 1 root root     60424 Mar 17 21:37  iconv
-rwxr-xr-x 1 root root     48064 Sep 24  2020  id
-rwxr-xr-x 1 root root     63720 Jan  1  2021  infocmp
lrwxrwxrwx 1 root root         3 Jan  1  2021  infotocap -> tic
-rwxr-xr-x 1 root root    159528 Sep 24  2020  install
-rwxr-xr-x 1 root root     30952 Jan 20 20:10  ionice
-rwxr-xr-x 1 root root     31024 Jan 20 20:10  ipcmk
-rwxr-xr-x 1 root root     35048 Jan 20 20:10  ipcrm
-rwxr-xr-x 1 root root     71912 Jan 20 20:10  ipcs
-rwxr-xr-x 1 root root     14608 Sep 27  2020  ischroot
-rwxr-xr-x 1 root root     56224 Sep 24  2020  join
-rwxr-xr-x 1 root root     47336 Jan 20 20:10  last
lrwxrwxrwx 1 root root         4 Jan 20 20:10  lastb -> last
-rwxr-xr-x 1 root root     32256 Feb  7  2020  lastlog
-rwxr-xr-x 1 root root      5401 Mar 17 21:37  ldd
-rwxr-xr-x 1 root root    175816 Jul  5  2020  less
-rwxr-xr-x 1 root root     39680 Sep 24  2020  link
lrwxrwxrwx 1 root root         7 Jan 20 20:10  linux32 -> setarch
lrwxrwxrwx 1 root root         7 Jan 20 20:10  linux64 -> setarch
-rwxr-xr-x 1 root root     55336 Mar 17 21:37  locale
-rwxr-xr-x 1 root root    315136 Mar 17 21:37  localedef
-rwxr-xr-x 1 root root     52040 Jan 20 20:10  logger
-rwxr-xr-x 1 root root     39712 Sep 24  2020  logname
-rwxr-xr-x 1 root root     14496 Jun  7  2021  lsattr
-rwxr-xr-x 1 root root    100584 Jan 20 20:10  lscpu
-rwxr-xr-x 1 root root     96488 Jan 20 20:10  lsipc
-rwxr-xr-x 1 root root     39480 Jan 20 20:10  lslocks
-rwxr-xr-x 1 root root     67816 Jan 20 20:10  lslogins
-rwxr-xr-x 1 root root     67816 Jan 20 20:10  lsmem
-rwxr-xr-x 1 root root     51440 Jan 20 20:10  lsns
-rwxr-xr-x 1 root root    158296 Feb 16  2020  mawk
-rwxr-xr-x 1 root root     35120 Jan 20 20:10  mcookie
-rwxr-xr-x 1 root root     48032 Sep 24  2020  md5sum
lrwxrwxrwx 1 root root         6 Sep 24  2020  md5sum.textutils -> md5sum
-rwxr-xr-x 1 root root     14568 Jan 20 20:10  mesg
-rwxr-xr-x 1 root root     68704 Sep 24  2020  mkfifo
-rwxr-xr-x 1 root root     35048 Jan 20 20:10  namei
lrwxrwxrwx 1 root root        22 Feb 28 00:00  nawk -> /etc/alternatives/nawk
-rwsr-xr-x 1 root root     44632 Feb  7  2020  newgrp
-rwxr-xr-x 1 root root     43840 Sep 24  2020  nice
-rwxr-xr-x 1 root root    113696 Sep 24  2020  nl
-rwxr-xr-x 1 root root     43872 Sep 24  2020  nohup
-rwxr-xr-x 1 root root     43840 Sep 24  2020  nproc
-rwxr-xr-x 1 root root     35280 Jan 20 20:10  nsenter
-rwxr-xr-x 1 root root     68576 Sep 24  2020  numfmt
-rwxr-xr-x 1 root root     72640 Sep 24  2020  od
lrwxrwxrwx 1 root root        23 Feb 28 00:00  pager -> /etc/alternatives/pager
-rwxr-xr-x 1 root root    121072 Jan 20 20:10  partx
-rwsr-xr-x 1 root root     63960 Feb  7  2020  passwd
-rwxr-xr-x 1 root root     43872 Sep 24  2020  paste
-rwxr-xr-x 1 root root     39712 Sep 24  2020  pathchk
-rwxr-xr-x 2 root root   3681152 Sep 24  2021  perl
-rwxr-xr-x 2 root root   3681152 Sep 24  2021  perl5.32.1
-rwxr-xr-x 1 root root     44000 Sep 24  2020  pinky
-rwxr-xr-x 1 root root     23112 Mar 17 21:37  pldd
-rwxr-xr-x 1 root root     76896 Sep 24  2020  pr
-rwxr-xr-x 1 root root     39680 Sep 24  2020  printenv
-rwxr-xr-x 1 root root     60256 Sep 24  2020  printf
-rwxr-xr-x 1 root root     39672 Jan 20 20:10  prlimit
-rwxr-xr-x 1 root root    138336 Sep 24  2020  ptx
-rwxr-xr-x 1 root root     56160 Sep 24  2020  realpath
-rwxr-xr-x 1 root root     14560 Jan 20 20:10  renice
lrwxrwxrwx 1 root root         4 Jan  1  2021  reset -> tset
-rwxr-xr-x 1 root root     63720 Jan 20 20:10  resizepart
-rwxr-xr-x 1 root root     14568 Jan 20 20:10  rev
-rwxr-xr-x 1 root root        30 Jan 29  2020  rgrep
-rwxr-xr-x 1 root root     43936 Sep 24  2020  runcon
-rwxr-xr-x 1 root root     10481 Sep 27  2020  savelog
-rwxr-xr-x 1 root root     67816 Jan 20 20:10  script
-rwxr-xr-x 1 root root     55528 Jan 20 20:10  scriptlive
-rwxr-xr-x 1 root root     43240 Jan 20 20:10  scriptreplay
-rwxr-xr-x 1 root root     48032 Jan  1  2021  sdiff
-rwxr-xr-x 1 root root     56192 Sep 24  2020  seq
-rwxr-xr-x 1 root root     27136 Jan 20 20:10  setarch
-rwxr-xr-x 1 root root     51440 Jan 20 20:10  setpriv
-rwxr-xr-x 1 root root     14568 Jan 20 20:10  setsid
-rwxr-xr-x 1 root root     47336 Jan 20 20:10  setterm
lrwxrwxrwx 1 root root         6 Feb  7  2020  sg -> newgrp
-rwxr-xr-x 1 root root     52128 Sep 24  2020  sha1sum
-rwxr-xr-x 1 root root     64416 Sep 24  2020  sha224sum
-rwxr-xr-x 1 root root     64416 Sep 24  2020  sha256sum
-rwxr-xr-x 1 root root     68512 Sep 24  2020  sha384sum
-rwxr-xr-x 1 root root     68512 Sep 24  2020  sha512sum
-rwxr-xr-x 1 root root     64608 Sep 24  2020  shred
-rwxr-xr-x 1 root root     60384 Sep 24  2020  shuf
-rwxr-xr-x 1 root root    118408 Sep 24  2020  sort
-rwxr-xr-x 1 root root     60896 Sep 24  2020  split
-rwxr-xr-x 1 root root     85184 Sep 24  2020  stat
-rwxr-xr-x 1 root root     52064 Sep 24  2020  stdbuf
-rwxr-xr-x 1 root root     48008 Sep 24  2020  sum
-rwxr-xr-x 1 root root     18584 Jan  1  2021  tabs
-rwxr-xr-x 1 root root    109536 Sep 24  2020  tac
-rwxr-xr-x 1 root root     76864 Sep 24  2020  tail
-rwxr-xr-x 1 root root     35048 Jan 20 20:10  taskset
-rwxr-xr-x 1 root root     43904 Sep 24  2020  tee
-rwxr-xr-x 1 root root     56128 Sep 24  2020  test
-rwxr-xr-x 1 root root     92432 Jan  1  2021  tic
-rwxr-xr-x 1 root root     44448 Sep 24  2020  timeout
-rwxr-xr-x 1 root root     22680 Jan  1  2021  toe
lrwxrwxrwx 1 root root        10 Feb 28 00:00  touch -> /bin/touch
-rwxr-xr-x 1 root root     26808 Jan  1  2021  tput
-rwxr-xr-x 1 root root     56160 Sep 24  2020  tr
-rwxr-xr-x 1 root root     43840 Sep 24  2020  truncate
-rwxr-xr-x 1 root root     30880 Jan  1  2021  tset
-rwxr-xr-x 1 root root     56128 Sep 24  2020  tsort
-rwxr-xr-x 1 root root     39712 Sep 24  2020  tty
-rwxr-xr-x 1 root root     15352 Mar 17 21:37  tzselect
-rwxr-xr-x 1 root root     43872 Sep 24  2020  unexpand
-rwxr-xr-x 1 root root     52128 Sep 24  2020  uniq
-rwxr-xr-x 1 root root     39680 Sep 24  2020  unlink
-rwxr-xr-x 1 root root     47576 Jan 20 20:10  unshare
-rwxr-xr-x 1 root root     59480 Apr 13  2021  update-alternatives
-rwxr-xr-x 1 root root     39744 Sep 24  2020  users
-rwxr-xr-x 1 root root     30952 Jan 20 20:10  utmpdump
-rwxr-sr-x 1 root tty      35048 Jan 20 20:10  wall
-rwxr-xr-x 1 root root     48072 Sep 24  2020  wc
-rwxr-xr-x 1 root root     31400 Jan 20 20:10  whereis
-rwxr-xr-x 1 root root       946 Sep 27  2020  which
-rwxr-xr-x 1 root root     60384 Sep 24  2020  who
-rwxr-xr-x 1 root root     39712 Sep 24  2020  whoami
lrwxrwxrwx 1 root root         7 Jan 20 20:10  x86_64 -> setarch
-rwxr-xr-x 1 root root     76152 Jan  9  2021  xargs
-rwxr-xr-x 1 root root     39680 Sep 24  2020  yes
-rwxr-xr-x 1 root root     22936 Mar 17 21:37  zdump
devent commented 2 years ago

I now have the Dockfile just apt-get install and the binaries are still missing.

FROM debian:bullseye-20220328
LABEL maintainer "Erwin Müller <erwin@muellerpublic.de>"

RUN set -x; \
    apt-get update && \
    apt-get install -y curl openssh-client gnupg gpg-agent git make && \
    rm -rf /var/lib/apt/lists/*

Log output: https://jenkins.anrisoftware.com/job/robobeerun-jenkins-helm-docker/job/Feature_4572/19/console Here is the image: https://harbor.anrisoftware.com/harbor/projects/2/repositories/jenkins-helm/artifacts/sha256:d00901453c28964a8d0388008ee206237c905914c6b84c1e7795987893a70d5f

achendev commented 2 years ago

Confirm related bug with missing in several cases only binaries or whole data directory in builded image in Kubernetes with Containerd runtime

Versions:

KUBERNETES-VERSION   OS-IMAGE          KERNEL-VERSION          CONTAINER-RUNTIME
v1.23.4              CentOS Stream 8   4.18.0-365.el8.x86_64   containerd://1.6.1

Manifest to reproduce missing whole data directory in Kubernetes

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-files
data:
  Dockerfile: |
    FROM node:16.14.0-alpine
    RUN apk add --no-cache python3
    WORKDIR /home/node/app
    COPY . ./
    RUN ls -laht /home/node/app
  env: |
    test=test
---
apiVersion: batch/v1
kind: Job
metadata:
  name: front-test-job
spec:
  template:
    metadata:
      name: front-dev-init
    spec:
      restartPolicy: Never
      containers:
        - name: buildimage
          image: gcr.io/kaniko-project/executor:latest
          args:
          - "--dockerfile=/Dockerfile"
          - "--context=dir:///home/node/app"
          - "--destination=myregistry:443/testrepo:test"
          resources: {}
          workingDir: /home/node/app
          volumeMounts:
            - name: code-volume
              mountPath: /home/node
            - name: test-files
              mountPath: /Dockerfile
              subPath: Dockerfile
      initContainers:
        - name: clone-repo
          command:
          - sh
          - -c
          - git clone https://github.com/Azure-Samples/html-docs-hello-world.git /home/node/app && cp /env /home/node/app/.env
          workingDir: /home/node/app
          image: alpine/git
          imagePullPolicy: Always
          resources: {}
          volumeMounts:
            - mountPath: /home/node
              name: code-volume
            - name: test-files
              mountPath: /env
              subPath: env
      volumes:
        - name: code-volume
          emptyDir: {}
        - name: test-files
          configMap:
            name: test-files

Check if data missed:

docker run  myregistry:443/testrepo:test ls /home/node/app

BUT if remove RUN apk add --no-cache python3 from Dockerfile and try to reproduce again, files will there:

docker run  myregistry:443/testrepo:test ls /home/node/app
LICENSE
README.md
css
fonts
img
index.html
js

but if i will build something for example with yarn some files could be missing too, but not whole directory

Pod log when files are missing:


INFO[0000] Retrieving image manifest node:16.14.0-alpine 
INFO[0000] Retrieving image node:16.14.0-alpine from registry index.docker.io 
INFO[0001] Built cross stage deps: map[]                
INFO[0001] Retrieving image manifest node:16.14.0-alpine 
INFO[0001] Returning cached image manifest              
INFO[0001] Executing 0 build triggers                   
INFO[0001] Unpacking rootfs as cmd RUN apk add --no-cache python3 requires it. 
INFO[0004] RUN apk add --no-cache python3               
INFO[0004] Taking snapshot of full filesystem...        
INFO[0005] cmd: /bin/sh                                 
INFO[0005] args: [-c apk add --no-cache python3]        
INFO[0005] Running: [/bin/sh -c apk add --no-cache python3] 
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
(1/11) Installing libbz2 (1.0.8-r1)
(2/11) Installing expat (2.4.7-r0)
(3/11) Installing libffi (3.4.2-r1)
(4/11) Installing gdbm (1.22-r0)
(5/11) Installing xz-libs (5.2.5-r1)
(6/11) Installing mpdecimal (2.5.1-r1)
(7/11) Installing ncurses-terminfo-base (6.3_p20211120-r0)
(8/11) Installing ncurses-libs (6.3_p20211120-r0)
(9/11) Installing readline (8.1.1-r0)
(10/11) Installing sqlite-libs (3.36.0-r0)
(11/11) Installing python3 (3.9.7-r4)
Executing busybox-1.34.1-r4.trigger
OK: 56 MiB in 27 packages
INFO[0006] Taking snapshot of full filesystem...        
INFO[0008] WORKDIR /home/node/app                       
INFO[0008] cmd: workdir                                 
INFO[0008] Changed working directory to /home/node/app  
INFO[0008] No files changed in this command, skipping snapshotting. 
INFO[0008] COPY . ./                                    
INFO[0008] Taking snapshot of files...                  
INFO[0008] RUN ls -laht /home/node/app                  
INFO[0008] cmd: /bin/sh                                 
INFO[0008] args: [-c ls -laht /home/node/app]           
INFO[0008] Running: [/bin/sh -c ls -laht /home/node/app] 
total 52K    
drwxr-xr-x    7 root     root        4.0K Apr 25 09:33 .
-rw-r--r--    1 root     root          10 Apr 25 09:33 .env
drwxr-xr-x    8 root     root        4.0K Apr 25 09:33 .git
-rw-r--r--    1 root     root        4.1K Apr 25 09:33 .gitignore
-rw-r--r--    1 root     root        1.2K Apr 25 09:33 LICENSE
-rw-r--r--    1 root     root         608 Apr 25 09:33 README.md
drwxr-xr-x    2 root     root        4.0K Apr 25 09:33 css
drwxr-xr-x    2 root     root        4.0K Apr 25 09:33 fonts
drwxr-xr-x    2 root     root        4.0K Apr 25 09:33 img
-rw-r--r--    1 root     root        1.7K Apr 25 09:33 index.html
drwxr-xr-x    2 root     root        4.0K Apr 25 09:33 js
drwxrwxrwx    3 root     root        4.0K Apr 25 09:33 ..
INFO[0008] Taking snapshot of full filesystem...        
INFO[0008] Pushing image to myregistry/testrepo:test 
INFO[0009] Pushed myregistry/testrepo@sha256:add254c2334297fbf12e26d9d5b7c6547010fc9a4a044f284a7e2d9bab56fb72

Reproducing only in Kubernetes, with docker it's okay Script example of the same Dockerfile with docker

docker run -d -p 5001:5000 --restart=always --name registry registry:2
mkdir ~/testkaniko/
echo test > ~/testkaniko/test
cat > ~/testkaniko/Dockerfile << 'EOL'
FROM node:16.14.0-alpine
RUN apk add --no-cache python3
WORKDIR /home/node/app
COPY . ./
RUN ls -laht /home/node/app
EOL
docker run --network="host" -v ~/testkaniko:/test gcr.io/kaniko-project/executor:latest --insecure --skip-tls-verify --dockerfile=/test/Dockerfile --context=dir:///test  --destination=127.0.0.1:5001/testrepo:test
docker run 127.0.0.1:5001/testrepo:test ls -laht /home/node/app

Docker kaniko log:

INFO[0000] Retrieving image manifest node:16.14.0-alpine
INFO[0000] Retrieving image node:16.14.0-alpine from registry index.docker.io
INFO[0001] Built cross stage deps: map[]
INFO[0001] Retrieving image manifest node:16.14.0-alpine
INFO[0001] Returning cached image manifest
INFO[0001] Executing 0 build triggers
INFO[0001] Unpacking rootfs as cmd RUN apk add --no-cache python3 requires it.
INFO[0005] RUN apk add --no-cache python3
INFO[0005] Taking snapshot of full filesystem...
INFO[0006] cmd: /bin/sh
INFO[0006] args: [-c apk add --no-cache python3]
INFO[0006] Running: [/bin/sh -c apk add --no-cache python3]
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
(1/11) Installing libbz2 (1.0.8-r1)
(2/11) Installing expat (2.4.7-r0)
(3/11) Installing libffi (3.4.2-r1)
(4/11) Installing gdbm (1.22-r0)
(5/11) Installing xz-libs (5.2.5-r1)
(6/11) Installing mpdecimal (2.5.1-r1)
(7/11) Installing ncurses-terminfo-base (6.3_p20211120-r0)
(8/11) Installing ncurses-libs (6.3_p20211120-r0)
(9/11) Installing readline (8.1.1-r0)
(10/11) Installing sqlite-libs (3.36.0-r0)
(11/11) Installing python3 (3.9.7-r4)
Executing busybox-1.34.1-r4.trigger
OK: 56 MiB in 27 packages
INFO[0008] Taking snapshot of full filesystem...
INFO[0010] WORKDIR /home/node/app
INFO[0010] cmd: workdir
INFO[0010] Changed working directory to /home/node/app
INFO[0010] Creating directory /home/node/app
INFO[0010] Taking snapshot of files...
INFO[0010] COPY . ./
INFO[0010] Taking snapshot of files...
INFO[0010] RUN ls -laht /home/node/app
INFO[0010] cmd: /bin/sh
INFO[0010] args: [-c ls -laht /home/node/app]
INFO[0010] Running: [/bin/sh -c ls -laht /home/node/app]
total 16K
drwxr-xr-x    2 root     root        4.0K Apr 25 09:51 .
drwxr-sr-x    3 node     node        4.0K Apr 25 09:51 ..
-rw-r--r--    1 root     root         117 Apr 25 09:51 Dockerfile
-rw-r--r--    1 root     root           5 Apr 25 09:51 test
INFO[0010] Taking snapshot of full filesystem...
INFO[0010] No files were changed, appending empty layer to config. No layer added to image.
INFO[0010] Pushing image to 127.0.0.1:5001/testrepo:test
INFO[0014] Pushed 127.0.0.1:5001/testrepo@sha256:ca91b9fcfdf84dd59d2b989c1b14c303e134b7270405f371ed228c1e3bd5a977

docker run 127.0.0.1:5001/testrepo:test ls -laht /home/node/app

total 16K
drwxr-xr-x    1 root     root        4.0K Apr 25 09:51 .
drwxr-sr-x    1 node     node        4.0K Apr 25 09:51 ..
-rw-r--r--    1 root     root         117 Apr 25 09:51 Dockerfile
-rw-r--r--    1 root     root           5 Apr 25 09:51 test

Files ok.

achendev commented 2 years ago

Succeeded to avoid this bug with changing context dir of Kaniko executor to make it different than WORKDIR in Dockerfile

Related to this issue: https://github.com/GoogleContainerTools/kaniko/issues/2021

Working manifest:

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-files
data:
  Dockerfile: |
    FROM node:16.14.0-alpine
    RUN apk add --no-cache python3
    WORKDIR /home/node/app
    COPY . ./
    RUN ls -laht /home/node/app
  env: |
    test=test
---
apiVersion: batch/v1
kind: Job
metadata:
  name: front-test-job
spec:
  template:
    metadata:
      name: front-dev-init
    spec:
      restartPolicy: Never
      containers:
        - name: buildimage
          image: gcr.io/kaniko-project/executor:latest
          args:
          - "--dockerfile=/Dockerfile"
          - "--context=dir:///code"
          - "--destination=myregistry:443/testrepo:test"
          resources: {}
          workingDir: /
          volumeMounts:
            - name: code-volume
              mountPath: /code
            - name: test-files
              mountPath: /Dockerfile
              subPath: Dockerfile
      initContainers:
        - name: clone-repo
          command:
          - sh
          - -c
          - git clone https://github.com/Azure-Samples/html-docs-hello-world.git /code && /bin/cp /env /code/.env
          workingDir: /code
          image: alpine/git
          imagePullPolicy: Always
          resources: {}
          volumeMounts:
            - name: code-volume
              mountPath: /code
            - name: test-files
              mountPath: /env
              subPath: env
      volumes:
        - name: code-volume
          emptyDir: {}
        - name: test-files
          configMap:
            name: test-files
headyj commented 2 years ago

Same issue here, also with nodejs alpine based image. But unfortunately changing the WORKDIR does not fix the issue

Is there any plan to fix this issue? this is quite annoying and unpredictable.

EDIT: I finally switched from an alpine-based to a debian-based nodejs image and it worked, but it's more a workaround than a solution...

gabyx commented 2 years ago

@achendev: Can you test with latest image on main: use the following instead of the ...:1.8.1 ...:1c0e5a0aca7f40f7a747cc89a365d0300015b06d

I think this is fixed.

jonathanmedd commented 2 years ago

We had the same issue too with missing binaries in /usr/bin when using a node alpine image, issue was not present in a node-lts image.

Have tested with the image mentioned above ...:1c0e5a0aca7f40f7a747cc89a365d0300015b06d and can confirm it's fixed for us.

Are you able to give an indication of how far away we are from getting it in a published kaniko release?