GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.73k stars 1.43k forks source link

kaniko multi stages Dockerfile #1558

Open mbeguet-bib opened 3 years ago

mbeguet-bib commented 3 years ago

Hello everyone. I'm working on a project who needs to use 2 images in the same Dockerfile. Unfortunately, with kaniko, it's difficult to support multi stage images. When I'm building the Dockerfile, I'm looses all changement during my first stage when i'm downloading my second stage. This is caused by : INFO[XXXX] Deleting filesystem...

Steps to reproduce the behavior:

  1. Dockerfile with 2 images for example

Additional Information

My Dockerfile :

FROM proactivehk/grails:2.4.5 as build
COPY myApplication myApplication
WORKDIR myApplication
RUN grails refresh-dependencies && grails dev war
FROM tomcat as base
COPY --from=build myApplication/target/XXXXX.war /opt/bitnami/tomcat/webapps_default

My script in Gitlab CI :

script:
    - export DOCKER_CONFIG=/kaniko/.docker/
    - export GOOGLE_APPLICATION_CREDENTIALS=/kaniko/.docker/config.json  
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY/dgd:$CI_JOB_ID

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
  • - [ ]
Please check if the build works in docker but not in kaniko
  • - [ ]
Please check if this error is seen when you use --cache flag
  • - [ ]
Please check if your dockerfile is a multistage dockerfile
  • - [ ]
dzirg44 commented 3 years ago

@Miguel1727 I am not so familiar with kaniko, but maybe cache options can deal with this problem?

mbeguet-bib commented 3 years ago

The path was not good. I've forgotten to add app/ before myApplication. Now, the correct path is : /app/myApplication/target/XXXXX.war

ashish0fficial commented 1 year ago

I am having similar problem but kaniko is failing to store the multistage Dockerfile

  FROM ubuntu AS base
  RUN echo "base"
  FROM base AS stage1
  RUN echo "stage1"
  CMD ["ls -lrth"]

Kaniko Image Dockerfile

FROM gcr.io/kaniko-project/executor:latest as kaniko

FROM busybox:latest as busybox

FROM jenkins/inbound-agent:latest-alpine
ENV JAVA_OPTS -XX:-UsePerfData
USER root

COPY --from=kaniko /kaniko /kaniko
COPY --from=busybox /bin /busybox

ENV PATH=/busybox:$PATH
ENV SSL_CERT_DIR /kaniko/ssl/certs
ENV PATH=/busybox:$PATH
ENV PATH=/busybox:$PATH
ENV DOCKER_CONFIG /kaniko/.docker/
ENV DOCKER_CREDENTIAL_GCR_CONFIG /kaniko/.config/gcloud/docker_credential_gcr_config.json
VOLUME /busybox
WORKDIR /kaniko

Below is the output of jenkins

13:12:10  + /kaniko/executor --context /home/jenkins/workspace/Junk/multistagedockerfile --dockerfile /home/jenkins/workspace/Junk/multistagedockerfile/Dockerfile --build-arg 'ENVIRONMENT=development' --verbosity debug --no-push --destination 192389191.dkr.ecr.ap-southeast-3.amazonaws.com/jenkins-kaniko-agent-development:41
13:12:10  DEBU[0000] Copying file /home/jenkins/workspace/Junk/multistagedockerfile/Dockerfile to /kaniko/Dockerfile 
13:12:10  DEBU[0000] file at /kaniko/Dockerfile already exists, resetting file ownership to root 
13:12:10  DEBU[0000] Skip resolving path /kaniko/Dockerfile       
13:12:10  DEBU[0000] Skip resolving path /home/jenkins/workspace/Junk/multistagedockerfile 
13:12:10  DEBU[0000] Skip resolving path /cache                   
13:12:10  DEBU[0000] Skip resolving path                          
13:12:10  DEBU[0000] Skip resolving path                          
13:12:10  DEBU[0000] Skip resolving path                          
13:12:10  DEBU[0000] Skip resolving path                          
13:12:10  INFO[0000] Resolved base name ubuntu to base            
13:12:10  INFO[0000] Resolved base name base to stage1            
13:12:10  DEBU[0000] Built stage name to index map: map[base:0 stage1:1] 
13:12:10  INFO[0000] Retrieving image manifest ubuntu             
13:12:10  INFO[0000] Retrieving image ubuntu from registry index.docker.io 
13:12:12  INFO[0002] Built cross stage deps: map[]                
13:12:12  INFO[0002] Retrieving image manifest ubuntu             
13:12:12  INFO[0002] Returning cached image manifest              
13:12:12  INFO[0002] Executing 0 build triggers                   
13:12:12  INFO[0002] Building stage 'ubuntu' [idx: '0', base-idx: '-1'] 
13:12:12  INFO[0002] Unpacking rootfs as cmd RUN echo "base" requires it. 
13:12:12  DEBU[0002] Ignore list: [{/kaniko false} {/etc/mtab false} {/tmp/apt-key-gpghome true} {/var/run false} {/proc false} {/dev false} {/dev/pts false} {/dev/shm false} {/dev/mqueue false} {/sys false} {/sys/fs/cgroup false} {/sys/fs/cgroup/systemd false} {/sys/fs/cgroup/perf_event false} {/sys/fs/cgroup/cpu,cpuacct false} {/sys/fs/cgroup/blkio false} {/sys/fs/cgroup/hugetlb false} {/sys/fs/cgroup/memory false} {/sys/fs/cgroup/cpuset false} {/sys/fs/cgroup/devices false} {/sys/fs/cgroup/pids false} {/sys/fs/cgroup/net_cls,net_prio false} {/sys/fs/cgroup/freezer false} {/busybox false} {/home/jenkins/.jenkins false} {/home/jenkins/agent false} {/etc/hosts false} {/etc/resolv.conf false} {/etc/hostname false} {/proc/bus false} {/proc/fs false} {/proc/irq false} {/proc/sys false} {/proc/sysrq-trigger false} {/proc/acpi false} {/proc/kcore false} {/proc/keys false} {/proc/latency_stats false} {/proc/timer_list false} {/proc/sched_debug false} {/sys/firmware false}] 
13:12:13  DEBU[0002] Not adding /dev because it is ignored        
13:12:13  DEBU[0002] Not adding /etc/hostname because it is ignored 
13:12:13  DEBU[0002] Not adding /etc/hosts because it is ignored  
13:12:13  DEBU[0002] Not adding /etc/resolv.conf because it is ignored 
13:12:13  DEBU[0002] Not adding /proc because it is ignored       
13:12:13  DEBU[0002] Not adding /sys because it is ignored        
13:12:14  DEBU[0004] Not adding /var/run because it is ignored    
13:12:14  INFO[0004] RUN echo "base"                              
13:12:14  INFO[0004] Initializing snapshotter ...                 
13:12:14  INFO[0004] Taking snapshot of full filesystem...        
13:12:15  DEBU[0004] Current image filesystem: map[]              
13:12:15  ERRO[0004] Couldn't eval /usr/lib/libssl.so.3 with link /usr/lib/libssl.so.3 
13:12:15  DEBU[0004] Adding to layer: [/ /lib /usr/lib /usr /usr/lib/ossl-modules /usr/lib/ossl-modules/legacy.so /usr/lib/modules-load.d /usr/lib/libssl.so.3] 
13:12:15  DEBU[0004] Deleting in layer: map[]                     
13:12:15  INFO[0004] Cmd: /bin/sh                                 
13:12:15  INFO[0004] Args: [-c echo "base"]                       
13:12:15  INFO[0004] Running: [/bin/sh -c echo "base"]            
13:12:15  base
13:12:15  INFO[0004] Taking snapshot of full filesystem...        
13:12:15  DEBU[0004] Current image filesystem: map[/:8635c3e565dc545b3680ea5bcbb8018324b94ae7287940fce56bb3fd6a18408b /lib:3fc1258b4c650de1583119d97232f2851d3e99ab62031a87a5d47f9e9bedad5f /usr:dc7596b23cc64594a9954571b5951e2a9c529849e279d091b6f6b8102891e470 /usr/lib:d1eb2d462eb516c8383942b24dde684b0f2446de8fe23a66edc4533274672658 /usr/lib/libssl.so.3:cf13375561cb47d163f185c510a46d4d9deb0ac63f749d1aed628be8e9a069e1 /usr/lib/modules-load.d:b3005600120f686f49531481ec60ef12fa01d62150eb4dd92024b290cc1084c8 /usr/lib/ossl-modules:d5764288c011e0261028afc7a52bed1c115865262f2a935d02411b505b5ed874 /usr/lib/ossl-modules/legacy.so:dc1c67ce0709c1554ca2201821ad4c66bcbb3cae0a49a54c5ac51ae1d0350a49] 
13:12:15  ERRO[0005] Couldn't eval /usr/lib/libcrypto.so.3 with link /usr/lib/libcrypto.so.3 
13:12:15  DEBU[0005] Adding to layer: [/usr/lib/engines-3 /usr/lib/engines-3/capi.so /usr/lib/engines-3/loader_attic.so /usr/lib/engines-3/padlock.so /usr/lib/engines-3/afalg.so /usr/lib/libcrypto.so.3] 
13:12:15  DEBU[0005] Deleting in layer: map[]                     
13:12:15  DEBU[0005] Mapping stage idx 0 to digest sha256:291969c6d2ffa46be786d9d81c8265feff88db491fd53600649188429512bf62 
13:12:15  DEBU[0005] Mapping digest sha256:291969c6d2ffa46be786d9d81c8265feff88db491fd53600649188429512bf62 to cachekey  
13:12:15  INFO[0005] Storing source image from stage 0 at path /kaniko/stages/0 
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // ecsTaskTemplate
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] wrap
[Pipeline] {
[Pipeline] sh
13:17:12  + echo Admin
13:17:12  Admin
[Pipeline] }
[Pipeline] // wrap
[Pipeline] slackSend
13:17:12  Slack Send Pipeline step running, values are - baseUrl: <empty>, teamDomain: avc, channel: #test, color: #FF0000, botUser: false, tokenCredentialId: slack-token, notifyCommitters: false, iconEmoji: <empty>, username: <empty>, timestamp: <empty>
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // ecsTaskTemplate
[Pipeline] End of Pipeline
ERROR: missing workspace /home/jenkins/workspace/Junk/multistagedockerfile on ecs-kaniko-kaniko-agent-axsna09
Finished: FAILURE

Its failed due to jenkins executor timeout with master, Whereas normal Dockerfile are working fine Example:

# syntax=docker/dockerfile:1

FROM node:18-alpine
ENV NODE_ENV=production

WORKDIR /app

# COPY ["package.json", "package-lock.json*", "./"]

RUN npm install sax

COPY . .

# CMD [ "node", "server.js" ]
Screenshot 2023-03-16 at 1 16 30 PM