error removing bin to make way for new symlink: unlinkat //bin/sh: permission denied

[willzhang]

Actual behavior build failed in jenkins docker pipeline

Expected behavior build success in jenkins docker pipeline

To Reproduce 1、create jenkinsfile

pipeline {
    agent any

    environment {
        GIT_URL = 'http://192.168.72.16:3001/root/demo1.git'
        GIT_CREDENTIALSID = 'gitea-account'
        GIT_BRANCH = 'main'
        BUILDER_IMAGE = '192.168.72.16/library/kaniko_executor:debug'
        CONTAINER_REGISTRY_URL = '192.168.72.16/library'
        CONTAINER_IMAGE_NAME = 'demo'
        CONFIG = credentials('kaniko-auth')
    }

    stages {
        stage('Clone') {
            steps {
                git branch: "${env.GIT_BRANCH}", credentialsId: "${env.GIT_CREDENTIALSID}", url: "${env.GIT_URL}"
            }
        }

        stage('Build_source') {
            agent {
                docker {
                    image 'docker.io/library/maven:latest'
                    args "-v $HOME/.m2:/root/.m2"
                }
            }
            steps {
                sh "mvn -v"
            }
        }

        stage('build_push_image') {
            agent {
                docker {
                    image "${env.BUILDER_IMAGE}"
                    args "--entrypoint="
                    reuseNode true
                }
            }
            steps {
                script {
                    withCredentials([file(credentialsId: 'kaniko-auth1', variable: 'DOCKER_CONFIG1')]) {
                        sh """
                            export DOCKER_CONFIG=/kaniko/docker/
                            mkdir -p /kaniko/docker/
                            cp $DOCKER_CONFIG1 /kaniko/docker/config.json
                            /kaniko/executor --context=dir://. --dockerfile=./Dockerfile --destination=${env.CONTAINER_REGISTRY_URL}/${env.CONTAINER_IMAGE_NAME}:${env.BUILD_NUMBER} --insecure
                        """
                    }
                }
            }
        }
    }
}

2、Dockerfile

FROM ubuntu
RUN apt update -y

3、jenkins console logs

main error

error building image: error building stage: failed to get filesystem from image: error removing bin to make way for new symlink: unlinkat //bin/sh: permission denied

all logs

Started by user [administrator](http://192.168.72.19:8080/user/admin)
[Pipeline] Start of Pipeline
[Pipeline] node
Running on [Jenkins](http://192.168.72.19:8080/computer/(built-in)/) in /var/lib/jenkins/workspace/docker-demo1
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $CONFIG
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Clone)
[Pipeline] git
The recommended git tool is: NONE
using credential gitea-account
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/docker-demo1/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://192.168.72.16:3001/root/demo1.git # timeout=10
Fetching upstream changes from http://192.168.72.16:3001/root/demo1.git
 > git --version # timeout=10
 > git --version # 'git version 2.34.1'
using GIT_ASKPASS to set credentials gitea-account
 > git fetch --tags --force --progress -- http://192.168.72.16:3001/root/demo1.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/main^{commit} # timeout=10
Checking out Revision f191facd01a657a8edee7e0fc91f89ca1669803e (refs/remotes/origin/main)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f191facd01a657a8edee7e0fc91f89ca1669803e # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D main # timeout=10
 > git checkout -b main f191facd01a657a8edee7e0fc91f89ca1669803e # timeout=10
Commit message: "更新 Dockerfile"
 > git rev-list --no-walk f191facd01a657a8edee7e0fc91f89ca1669803e # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build_source)
[Pipeline] node
Running on [Jenkins](http://192.168.72.19:8080/computer/(built-in)/) in /var/lib/jenkins/workspace/docker-demo1@2
[Pipeline] {
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . docker.io/library/maven:latest
.
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 115:120 -v /var/lib/jenkins/.m2:/root/.m2 -w /var/lib/jenkins/workspace/docker-demo1@2 -v /var/lib/jenkins/workspace/docker-demo1@2:/var/lib/jenkins/workspace/docker-demo1@2:rw,z -v /var/lib/jenkins/workspace/docker-demo1@2@tmp:/var/lib/jenkins/workspace/docker-demo1@2@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** docker.io/library/maven:latest cat
$ docker top 6e47652297ce0e2c66f4cdbfc605dae06b3cc7262152836b431c8f99bea37e6a -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ mvn -v
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: /usr/share/maven
Java version: 21.0.2, vendor: Eclipse Adoptium, runtime: /opt/java/openjdk
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-105-generic", arch: "amd64", family: "unix"
[Pipeline] }
$ docker stop --time=1 6e47652297ce0e2c66f4cdbfc605dae06b3cc7262152836b431c8f99bea37e6a
$ docker rm -f --volumes 6e47652297ce0e2c66f4cdbfc605dae06b3cc7262152836b431c8f99bea37e6a
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (build_push_image)
[Pipeline] getContext
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . 192.168.72.16/library/kaniko_executor:debug
.
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 115:120 --entrypoint= -w /var/lib/jenkins/workspace/docker-demo1 -v /var/lib/jenkins/workspace/docker-demo1:/var/lib/jenkins/workspace/docker-demo1:rw,z -v /var/lib/jenkins/workspace/docker-demo1@tmp:/var/lib/jenkins/workspace/docker-demo1@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** 192.168.72.16/library/kaniko_executor:debug cat
$ docker top fb45030f7345d7cac091ca04250320c6a01ae721410b8d3ada6d0c1fe66e2c51 -eo pid,comm
[Pipeline] {
[Pipeline] script
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKER_CONFIG1
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
         Affected argument(s) used the following variable(s): [DOCKER_CONFIG1]
         See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ export 'DOCKER_CONFIG=/kaniko/docker/'
+ mkdir -p /kaniko/docker/
+ echo /var/lib/jenkins
/var/lib/jenkins
+ pwd
/var/lib/jenkins/workspace/docker-demo1
+ ls
Dockerfile
README.md
+ cp **** /kaniko/docker/config.json
+ /kaniko/executor '--context=dir://.' '--dockerfile=./Dockerfile' '--destination=192.168.72.16/library/demo:59' --insecure
INFO[0000] Retrieving image manifest ubuntu             
INFO[0000] Retrieving image ubuntu from registry index.docker.io 
INFO[0006] Built cross stage deps: map[]                
INFO[0006] Retrieving image manifest ubuntu             
INFO[0006] Returning cached image manifest              
INFO[0006] Executing 0 build triggers                   
INFO[0006] Building stage 'ubuntu' [idx: '0', base-idx: '-1'] 
INFO[0006] Unpacking rootfs as cmd RUN apt update -y requires it. 
error building image: error building stage: failed to get filesystem from image: error removing bin to make way for new symlink: unlinkat //bin/sh: permission denied
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // script
[Pipeline] }
$ docker stop --time=1 fb45030f7345d7cac091ca04250320c6a01ae721410b8d3ada6d0c1fe66e2c51
$ docker rm -f --volumes fb45030f7345d7cac091ca04250320c6a01ae721410b8d3ada6d0c1fe66e2c51
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE

The debug image i use with image id c7c1f8d3d464

root@jenkins:/var/lib/jenkins/workspace# docker images |grep debug
192.168.72.16/library/kaniko_executor   debug                c7c1f8d3d464   4 weeks ago    99.1MB

[ROunofF]

I just ran into the same error you described: error building image: error building stage: failed to get filesystem from image: error removing bin to make way for new symlink: unlinkat //bin/sh: permission denied and my problem was that my kaniko pods needed to run as root.

Check if your pipeline is making it running as root here? Since apt-get is a root command, its needs root to be succesful.

[Max-LAB]

I also just run into sam error. I have runner deployed in Open Shift runner through Gitlab Runner Operator. This is wahat I am trying to do

build_image:
  stage: build
  tags:
    - k8s
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - /kaniko/executor 
      --context $CI_PROJECT_DIR
      --dockerfile $CI_PROJECT_DIR/Dockerfile
      --no-push
      --verbosity=trace
      --no-push-cache

I am stuck on it, and this is the onyl place that reference the same error. @ROunofF can you explain exactly what did you do? Thanks!