search

GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.88k stars 1.44k forks source link

Kaniko with Remote ECR repo for --cache-repo #3089

Open sushama-kothawale opened 7 months ago

sushama-kothawale commented 7 months ago

This is related to Kaniko do not use cache options

When we set --cache-repo as ECR repo url, kaniko push all layers to ecr repo as cache, if dockerfile has too many/multi-step instructions, this increases the ECR repo storage size. And for each build it pushes the cache to ecr repo. Can we have option like if layer cache is present in ECR repo then use that cache while building the image and do not push it again?

I am using below options to build image with kaniko:

docker run gcr.io/kaniko-project/executor:latest --context=/workspace/build --dockerfile=/workspace/build/{dockerfile} --destination={image_tag_version} --cache=true --cache-repo=84592232.dkr.ecr.ap-south-1.amazonaws.com/dev-builds --use-new-run --cache-run-layers --cache-copy-layers --snapshot-mode=redo --cleanup --verbosity=info --log-timestamp

our build requirement is daily 20-30 build of different code languages.. ECR has the storage limit of 10000 tags for each repo. So limit exceeds within 2-3 days.

Can someone check this?

sushama-kothawale commented 7 months ago

@jaugustin @aaron-prindle @JeromeJu @QuanZhang-William It seems you marked this as feature request? Can you please confirm if this is the case? or any argument I need to pass while docker run command to achieve the same?

sushama-kothawale commented 7 months ago

@QuanZhang-William any updates on this? We are blocked on this for integrating kaniko with our prod builds..

mzihlmann commented 1 month ago

could you verify you're not running into this issue? https://github.com/GoogleContainerTools/kaniko/issues/3340

with --cache only RUN instructions are cached. with cache-copy-layers only COPY instructions are cached. So layers created by WORKDIR or ADD will cause cache misses. This is not easily noticeable when doing single stage builds, but pops up clearly when doing multistage ones.