GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.87k stars 1.44k forks source link

error if kaniko takes longer than 1 hour when running on AWS IRSA #3199

Open tooptoop4 opened 5 months ago

tooptoop4 commented 5 months ago

Actual behavior when the kaniko step (run from a pod using AWS IRSA) takes over 1 hour it fails with this error

INFO[3716] Pushing image to redact.amazonaws.com/redact.ecr/redact:0.0.72 error pushing image: failed to push to destination redact.amazonaws.com/redact.ecr/redact:0.0.72: GET https://public.ecr.aws/v2/docker/library/python/blobs/sha256:d2c04aca259ccbbbd92a78c0452532b76b5b2812b06999bafaaae910297770a9: DENIED: Your Authorization Token is invalid.

Expected behavior Image is able to be built and pushed to ECR

To Reproduce Steps to reproduce the behavior:

  1. I am running on EKS with IRSA and trying to push to ECR
  2. if the kaniko command (/kaniko/executor --context /repo/redact --dockerfile /repo/redact/Dockerfile --destination=redact) takes just over 1 hour then it fails

Additional Information

env variables:

aaron-prindle commented 4 months ago

Related: https://github.com/GoogleContainerTools/kaniko/issues/2526