search

redhat-buildpacks / poc

Project to be used to perform go build of images using Buildah, ...
3 stars 6 forks source link

kaniko/pkg/util.mkdirAllWithPermissions #9

Closed cmoulliard closed 2 years ago

cmoulliard commented 2 years ago

Issue

When we use a red hat ubi8 image (= RHL OS), then the following error is reported by kaniko during the step to build the following dockerfile

FROM registry.access.redhat.com/ubi8-minimal
RUN microdnf install -y wget python3

Error

INFO[0000] Building the Dockerfile2
DEBU[0000] Moving to kaniko home dir: /kaniko
DEBU[0000] Building the Dockerfile2 ...
DEBU[0000] Built stage name to index map: map[]
INFO[0000] Retrieving image manifest redhat/ubi8:latest
INFO[0000] Retrieving image redhat/ubi8:latest from registry index.docker.io
INFO[0001] Built cross stage deps: map[]
INFO[0001] Retrieving image manifest redhat/ubi8:latest
INFO[0001] Returning cached image manifest
INFO[0001] Executing 0 build triggers
INFO[0001] Unpacking rootfs as cmd COPY ./hello.txt /workspace requires it.
DEBU[0001] Mounted directories: [{/kaniko false} {/etc/mtab false} {/tmp/apt-key-gpghome true} {/proc false} {/dev false} {/dev/pts false} {/dev/mqueue false} {/sys false} {/sys/fs/cgroup false} {/sys/fs/cgroup/cpuset false} {/sys/fs/cgroup/cpu false} {/sys/fs/cgroup/cpuacct false} {/sys/fs/cgroup/blkio false} {/sys/fs/cgroup/memory false} {/sys/fs/cgroup/devices false} {/sys/fs/cgroup/freezer false} {/sys/fs/cgroup/net_cls false} {/sys/fs/cgroup/perf_event false} {/sys/fs/cgroup/net_prio false} {/sys/fs/cgroup/hugetlb false} {/sys/fs/cgroup/pids false} {/sys/fs/cgroup/rdma false} {/sys/fs/cgroup/systemd false} {/workspace false} {/cache false} {/etc/hosts false} {/dev/termination-log false} {/etc/hostname false} {/etc/resolv.conf false} {/dev/shm false} {/run/secrets/kubernetes.io/serviceaccount false} {/proc/bus false} {/proc/fs false} {/proc/irq false} {/proc/sys false} {/proc/sysrq-trigger false} {/proc/acpi false} {/proc/kcore false} {/proc/keys false} {/proc/timer_list false} {/proc/sched_debug false} {/sys/firmware false}]
DEBU[0026] Not adding /dev because it is ignored
DEBU[0026] Not adding /etc/resolv.conf because it is ignored
DEBU[0026] Not adding /etc/hosts because it is ignored
DEBU[0027] Not adding /etc/hostname because it is ignored
DEBU[0027] Not adding /etc/mtab because it is ignored
DEBU[0027] Not adding /proc because it is ignored
DEBU[0027] Not adding /sys because it is ignored
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xfa90aa]
...
goroutine 1 [running]:
goroutine 1 :                                                                                                                                  
github.com/GoogleContainerTools/kaniko/pkg/util.mkdirAllWithPermissions(0xc000171ae0, 0xf, 0xc0800001fd, 0x0, 0xc, 0x0, 0x0)                   
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/util/fs_util.go:763 +0x5ea                                            
github.com/GoogleContainerTools/kaniko/pkg/util.ExtractFile(0x248a4a8, 0x1, 0xc0009bce00, 0x24a41e0, 0xc00029c480, 0x0, 0x0)                   
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/util/fs_util.go:335 +0x1a78                                           
github.com/GoogleContainerTools/kaniko/pkg/util.GetFSFromLayers(0x248a4a8, 0x1, 0xc000621400, 0x2, 0x2, 0xc00072e860, 0x1, 0x1, 0x0, 0x0, ...) 
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/util/fs_util.go:196 +0xf3c                                            
github.com/GoogleContainerTools/kaniko/pkg/util.GetFSFromImage(0x248a4a8, 0x1, 0x24f2888, 0xc00040e1c0, 0x23625c0, 0x0, 0x0, 0x0, 0x0, 0x0)    
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/util/fs_util.go:132 +0x265                                            
github.com/GoogleContainerTools/kaniko/pkg/executor.(*stageBuilder).build(0xc000221600, 0x0, 0x0)                                              
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/executor/build.go:305 +0x610                                          
github.com/GoogleContainerTools/kaniko/pkg/executor.DoBuild(0xc0002891d0, 0x0, 0x0, 0x0, 0x0)                                                  
    /go/src/kaniko-app/vendor/github.com/GoogleContainerTools/kaniko/pkg/executor/build.go:594 +0x9a5
cmoulliard commented 2 years ago

The panic error is returned when /var/spool/mail is processed by github.com/GoogleContainerTools/kaniko/pkg/util.mkdirAllWithPermissions at line 776

func mkdirAllWithPermissions(path string, mode os.FileMode, uid, gid int64) error {
    // Check if a file already exists on the path, if yes then delete it
    info, err := os.Stat(path)
    if !os.IsNotExist(err) && !info.IsDir() {

Logging Trace

TRAC[0001] symlink from spool/mail to /var/mail         
TRAC[0001] creating dir /var/gopher                     
TRAC[0001] creating dir /var/preserve                   
TRAC[0001] creating dir /var/cache                      
TRAC[0001] creating dir /var/spool                      
TRAC[0001] creating dir /var/spool/mail
Screenshot 2021-11-21 at 17 18 19
cmoulliard commented 2 years ago

What is really strange is that we dont experiment such an error using the kaniko v1.7 image

FROM registry.access.redhat.com/ubi8-minimal

COPY ./hello.txt /workspace

LABEL "website.name"="geeksforgeeks website"

RUN microdnf install -y wget python3

and 

docker run \
    -v $(pwd)/workspace:/workspace \
    gcr.io/kaniko-project/executor:latest \
    --dockerfile /workspace/ubi8-minimal \
    --context dir:///workspace/ \
    --no-push
...
docker run \                                    
    -v $(pwd)/workspace:/workspace \
    gcr.io/kaniko-project/executor:latest \
    --dockerfile /workspace/ubi8-minimal \
    --context dir:///workspace/ \
    --no-push
INFO[0000] Retrieving image manifest registry.access.redhat.com/ubi8-minimal 
INFO[0000] Retrieving image registry.access.redhat.com/ubi8-minimal from registry registry.access.redhat.com 
INFO[0000] Built cross stage deps: map[]                
INFO[0000] Retrieving image manifest registry.access.redhat.com/ubi8-minimal 
INFO[0000] Returning cached image manifest              
INFO[0000] Executing 0 build triggers                   
INFO[0000] Unpacking rootfs as cmd COPY ./hello.txt /workspace requires it. 
INFO[0023] COPY ./hello.txt /workspace                  
INFO[0023] Taking snapshot of files...                  
INFO[0023] LABEL "website.name"="geeksforgeeks website" 
INFO[0023] Applying label website.name=geeksforgeeks website 
INFO[0023] RUN microdnf install -y wget python3         
INFO[0023] Taking snapshot of full filesystem...        
INFO[0023] cmd: /bin/sh                                 
INFO[0023] args: [-c microdnf install -y wget python3]  
INFO[0023] Running: [/bin/sh -c microdnf install -y wget python3] 
...
Installing: wget;1.19.5-10.el8;x86_64;ubi-8-appstream
Complete.
INFO[0037] Taking snapshot of full filesystem...        
...
INFO[0038] Skipping push to container registry due to --no-push flag
matejvasek commented 2 years ago

@cmoulliard The issues seems to be in alpine image as a runner. It has /var/spool/mail -> /var/mail symlink. And the ubi8-minimal has symlink /var/mail -> spool/mail. I think that while copying root FS cyclical symlink is creaded. The official kaniko image uses scratch so it won't happen.

matejvasek commented 2 years ago

When I used alpine:3 (instead of scratch) as a base for standard gcr.io/kaniko-project/executor image it panics the same way.

cmoulliard commented 2 years ago

When I used alpine:3 (instead of scratch) as a base for standard gcr.io/kaniko-project/executor image it panics the same way.

Can we then create an image using scratch where Dockerfile is defined as ?

FROM scratch

COPY --from=debugger /go/bin/dlv /usr/local/bin
COPY --from=builder /kaniko-app /kaniko-app
COPY --from=builder /go/src/kaniko-app/entrypoint.sh /entrypoint.sh

RUN chmod +x /entrypoint.sh

RUN mkdir -p /kaniko /workspace /cache
WORKDIR /workspace

ENTRYPOINT ["/entrypoint.sh"]

Remark: Existing Dockerfile to build the kaniko build app is - https://github.com/redhat-buildpacks/poc/blob/cb7966371c4517a486280630950e0dbac477de73/kaniko/Dockerfile_build#L24-L35

@matejvasek

matejvasek commented 2 years ago

Merely updating FROM clause won't work as for instance chmod mkdir or shell is not present in scratch image. But it should be doable somehow.

matejvasek commented 2 years ago

I can create PR for that.

natalieparellano commented 2 years ago

FWIW I've had good luck with distroless/static as the runner.