kaniko fails to build Dockerfile inside kubernetes pod but can build with docker run...

[bharath-krishna]

Actual behavior

• I have created a kaniko pod (image: gcr.io/kaniko-project/executor:v1.7.0-debug) with /busybox/sleep infinity command
• Switched to to container shell with kubectl exec command
• Created a Dockerfile in /workspace dir (contents below)
• Ran executor as below
• executor --dockerfile=Dockerfile --no-push

• Getting error as below

  /workspace # executor --dockerfile=Dockerfile --no-push INFO[0000] Retrieving image manifest python:3.7.6-alpine3.11 INFO[0000] Retrieving image python:3.7.6-alpine3.11 from registry index.docker.io INFO[0002] Built cross stage deps: map[]
  INFO[0002] Retrieving image manifest python:3.7.6-alpine3.11 INFO[0002] Returning cached image manifest
  INFO[0002] Executing 0 build triggers
  INFO[0002] Unpacking rootfs as cmd RUN apk add --no-cache openssl curl ca-certificates ssmtp requires it. INFO[0005] RUN apk add --no-cache openssl curl ca-certificates ssmtp INFO[0005] Taking snapshot of full filesystem...
  . . . (14/15) Installing perl-git (2.24.4-r0) (15/15) Installing git-perl (2.24.4-r0) Executing busybox-1.31.1-r9.trigger ERROR: busybox-1.31.1-r9.trigger: script exited with error 127 1 error; 102 MiB in 55 packages error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1

Expected behavior Expected to get output as below when I run kaniko with docker run command

docker run -it --rm -v `pwd`:/workspace gcr.io/kaniko-project/executor:v1.7.0-debug --dockerfile=Dockerfile --no-push
INFO[0000] Retrieving image manifest python:3.7.6-alpine3.11 
INFO[0000] Retrieving image python:3.7.6-alpine3.11 from registry index.docker.io 
.
.
.
OK: 102 MiB in 55 packages
INFO[0017] Taking snapshot of full filesystem...        
INFO[0019] Skipping push to container registry due to --no-push flag 

To Reproduce Steps to reproduce the behavior:

1. Create a kubernetes pod with kaniko debug image and /busybox/sleep infinity command
2. Switch to container terminal with kubectl exec command
3. Create a Dockerfile in /workspace directory
4. Run command executor --dockerfile=Dockerfile --no-push

Additional Information

• Dockerfile

  FROM python:3.7.6-alpine3.11
  
  RUN apk add --no-cache openssl curl ca-certificates ssmtp
  
  # Install the python package that is necessary
  RUN apk add --no-cache git cmake perl

• Build Context Nothing else only Docker file

• Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:v1.7.0-debug

• Kubernetes container commands and args command = ["/bin/sh", "-c"] args: [ "/busybox/sleep infinity" ]

  Triage Notes for the Maintainers

  Description	Yes/No
  Please check if this a new feature you are proposing	- [ ]
  Please check if the build works in docker but not in kaniko	- [x]
  Please check if this error is seen when you use --cache flag	- [ ]
  Please check if your dockerfile is a multistage dockerfile	- [ ]

[doriandekoning]

We have a similar issue (on 1.6.0, 1.7.0 and 1.8.1), since it has been quite a while did you find a solution to this problem (or is there another update maybe)?

[bharath-krishna]

It's been quite a while, and if I remember correctly, the solution we found out was not to change USER in Dockerfile frequently, I mean changing from APP user to ROOT user and back.

If there are multiple changes like this then Kaniko fails to build.

[doriandekoning]

Thanks for the reply! We did not do any user changes so that was a problem for us. When we switched to another base image (busybox -> debian) we did not encounter the problem anymore.