search

GoogleContainerTools / skaffold

Easy and Repeatable Kubernetes Development
https://skaffold.dev/
Apache License 2.0
14.81k stars 1.61k forks source link

Cannot update mutliple containers created from single image #9392

Open miladrahimi opened 2 months ago

miladrahimi commented 2 months ago

Expected behavior

When I have two (multiple) containers from an image, I expect both containers to be updates on code change in development environment.

Actual behavior

It updates one container and for the other one raises this error:

INFO[0406] files modified: [app/src/ControlApi/Handlers/MyHandler.php]  subtask=-1 task=DevLoop
Syncing 1 files for 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:448f38d9c0ea842e2f95da8c5e18f37bafea9bca0759fface0520ff391d1990d
INFO[0406] Copying files:map[app/src/ControlApi/Handlers/MyHandler.php:[/var/www/app/src/ControlApi/Handlers/MyHandler.php]]to040672967332.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:448f38d9c0ea842e2f95da8c5e18f37bafea9bca0759fface0520ff391d1990d  subtask=-1 task=DevLoop
WARN[0407] Skipping deploy due to sync error:copying files: running [kubectl --context docker-desktop --namespace project exec project-sandbox-backend-69fbf9899c-6zhjb --namespace project -c backend -i -- tar xmf - -C / --no-same-owner]
 - stdout: ""
 - stderr: "tar: Removing leading `/' from member names\ntar: var/www/app/src/ControlApi/Handlers/MyHandler.php: Cannot open: File exists\ntar: Exiting with failure status due to previous errors\ncommand terminated with exit code 2\n"
 - cause: exit status 2: sync failed for artifact "12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:448f38d9c0ea842e2f95da8c5e18f37bafea9bca0759fface0520ff391d1990d"  subtask=-1 task=DevLoop
Watching for changes...

Information

apiVersion: skaffold/v2beta9
kind: Config
metadata:
  name: project
build:
  artifacts:
    - image: 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend
      context: app
      docker:
        buildArgs:
          TARGET: "DEV"
          WITH_XDEBUG: '{{.WITH_XDEBUG}}'
      sync:
        manual:
        - src: src/**/*
          dest: src/
          strip: src/
        - src: tests/**/*
          dest: tests/
          strip: tests/
        - src: templates/**/*.twig
          dest: templates/
          strip: templates/
        - src: config/**/*.yaml
          dest: config/
          strip: config/
        - src: public/**/*
          dest: public/
          strip: public/
    - image: 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/frontend
      context: frontend
      docker:
        dockerfile: Dockerfile
      sync:
        manual:
          - src: pages/**/*
            dest: pages/
            strip: pages/
          - src: components/**/*
            dest: components/
            strip: components/
          - src: styles/**/*
            dest: styles/
            strip: styles/
          - src: public/**/*
            dest: public/
            strip: public/
          - src: src/**/*
            dest: src/
            strip: src/
  tagPolicy:
    sha256: {}

  local:
    concurrency: 4
    useBuildkit: true
    push: false

deploy:
  helm:
    releases:
      - name: "project"
        chartPath: "helm-charts/project"
        skipBuildDependencies: true
        artifactOverrides:
          backend.image: 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend
          frontend.image: 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/frontend
        valuesFiles:
          - helm-charts/project/values.yaml
          - helm-charts/project/values-dev.yaml
          - helm-charts/project/values-local.yaml

Steps to reproduce the behavior

  1. a clonable repository with the sample skaffold project
  2. skaffold dev -n project -v INFO --port-forward --kube-context=docker-desktop
  3. change a file to update two or more containers which built on a image created by Skaffold
idsulik commented 2 months ago

@miladrahimi hi! could you please try it in debug mode and provide the logs?
skaffold dev -n project -v debug --port-forward --kube-context=docker-desktop.

miladrahimi commented 2 months ago

This is the debug mode output:

DEBU[0045] Change detectednotify.Write: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Write: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Create: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Write: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Create: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Write: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Remove: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0045] Change detectednotify.Remove: "/Users/milad.rahimi/Projects/project/app/src/ControlApi/Handlers/MyHandler.php~"  subtask=-1 task=DevLoop
DEBU[0046] Executing template &{envTemplate 0x140019c8360 0x1400135a9b0  } with environment map[COMMAND_MODE:unix2003 HOME:/Users/milad.rahimi HOMEBREW_CELLAR:/opt/homebrew/Cellar HOMEBREW_PREFIX:/opt/homebrew HOMEBREW_REPOSITORY:/opt/homebrew IDEA_INITIAL_DIRECTORY:/ INFOPATH:/opt/homebrew/share/info: LC_CTYPE:UTF-8 LESS:-R LOGNAME:milad.rahimi LSCOLORS:Gxfxcxdxbxegedabagacad LS_COLORS:di=1;36:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43:su=30;41:sg=30;46:tw=30;42:ow=30;43 MANPATH:/opt/homebrew/share/man:: OLDPWD:/Users/milad.rahimi/Projects/project PAGER:less PATH:/opt/homebrew/Cellar/php/8.2.8/bin:/opt/homebrew/opt/php@8.1/sbin:/opt/homebrew/opt/php@8.1/bin:/Users/milad.rahimi/bin:/Users/milad.rahimi/Scripts/ngrock:/Users/milad.rahimi/Scripts/google-cloud-sdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/milad.rahimi/Library/Application Support/JetBrains/Toolbox/scripts PWD:/Users/milad.rahimi/Projects/project SHELL:/bin/zsh SHLVL:1 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.8R8b4RkUYk/Listeners TERM:xterm-256color TERMINAL_EMULATOR:JetBrains-JediTerm TERM_SESSION_ID:06ef3b59-ab21-418c-9b87-0546ac49ff1b TMPDIR:/var/folders/b4/t_26_kdx7sq4x099mj49g4x00000gq/T/ USER:milad.rahimi XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/milad.rahimi/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.jetbrains.PhpStorm __CF_USER_TEXT_ENCODING:0x1F7:0x0:0x0]  subtask=-1 task=DevLoop
DEBU[0046] Executing template &{envTemplate 0x140019c9200 0x1400135aa00  } with environment map[COMMAND_MODE:unix2003 HOME:/Users/milad.rahimi HOMEBREW_CELLAR:/opt/homebrew/Cellar HOMEBREW_PREFIX:/opt/homebrew HOMEBREW_REPOSITORY:/opt/homebrew IDEA_INITIAL_DIRECTORY:/ INFOPATH:/opt/homebrew/share/info: LC_CTYPE:UTF-8 LESS:-R LOGNAME:milad.rahimi LSCOLORS:Gxfxcxdxbxegedabagacad LS_COLORS:di=1;36:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43:su=30;41:sg=30;46:tw=30;42:ow=30;43 MANPATH:/opt/homebrew/share/man:: OLDPWD:/Users/milad.rahimi/Projects/project PAGER:less PATH:/opt/homebrew/Cellar/php/8.2.8/bin:/opt/homebrew/opt/php@8.1/sbin:/opt/homebrew/opt/php@8.1/bin:/Users/milad.rahimi/bin:/Users/milad.rahimi/Scripts/ngrock:/Users/milad.rahimi/Scripts/google-cloud-sdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/milad.rahimi/Library/Application Support/JetBrains/Toolbox/scripts PWD:/Users/milad.rahimi/Projects/project SHELL:/bin/zsh SHLVL:1 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.8R8b4RkUYk/Listeners TERM:xterm-256color TERMINAL_EMULATOR:JetBrains-JediTerm TERM_SESSION_ID:06ef3b59-ab21-418c-9b87-0546ac49ff1b TMPDIR:/var/folders/b4/t_26_kdx7sq4x099mj49g4x00000gq/T/ USER:milad.rahimi XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/milad.rahimi/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.jetbrains.PhpStorm __CF_USER_TEXT_ENCODING:0x1F7:0x0:0x0]  subtask=-1 task=DevLoop
DEBU[0046] Executing template &{envTemplate 0x14001a09680 0x1400135bd60  } with environment map[COMMAND_MODE:unix2003 HOME:/Users/milad.rahimi HOMEBREW_CELLAR:/opt/homebrew/Cellar HOMEBREW_PREFIX:/opt/homebrew HOMEBREW_REPOSITORY:/opt/homebrew IDEA_INITIAL_DIRECTORY:/ INFOPATH:/opt/homebrew/share/info: LC_CTYPE:UTF-8 LESS:-R LOGNAME:milad.rahimi LSCOLORS:Gxfxcxdxbxegedabagacad LS_COLORS:di=1;36:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43:su=30;41:sg=30;46:tw=30;42:ow=30;43 MANPATH:/opt/homebrew/share/man:: OLDPWD:/Users/milad.rahimi/Projects/project PAGER:less PATH:/opt/homebrew/Cellar/php/8.2.8/bin:/opt/homebrew/opt/php@8.1/sbin:/opt/homebrew/opt/php@8.1/bin:/Users/milad.rahimi/bin:/Users/milad.rahimi/Scripts/ngrock:/Users/milad.rahimi/Scripts/google-cloud-sdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/milad.rahimi/Library/Application Support/JetBrains/Toolbox/scripts PWD:/Users/milad.rahimi/Projects/project SHELL:/bin/zsh SHLVL:1 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.8R8b4RkUYk/Listeners TERM:xterm-256color TERMINAL_EMULATOR:JetBrains-JediTerm TERM_SESSION_ID:06ef3b59-ab21-418c-9b87-0546ac49ff1b TMPDIR:/var/folders/b4/t_26_kdx7sq4x099mj49g4x00000gq/T/ USER:milad.rahimi XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/milad.rahimi/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.jetbrains.PhpStorm __CF_USER_TEXT_ENCODING:0x1F7:0x0:0x0]  subtask=-1 task=DevLoop
DEBU[0046] Executing template &{envTemplate 0x140016b2120 0x1400135bdb0  } with environment map[COMMAND_MODE:unix2003 HOME:/Users/milad.rahimi HOMEBREW_CELLAR:/opt/homebrew/Cellar HOMEBREW_PREFIX:/opt/homebrew HOMEBREW_REPOSITORY:/opt/homebrew IDEA_INITIAL_DIRECTORY:/ INFOPATH:/opt/homebrew/share/info: LC_CTYPE:UTF-8 LESS:-R LOGNAME:milad.rahimi LSCOLORS:Gxfxcxdxbxegedabagacad LS_COLORS:di=1;36:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43:su=30;41:sg=30;46:tw=30;42:ow=30;43 MANPATH:/opt/homebrew/share/man:: OLDPWD:/Users/milad.rahimi/Projects/project PAGER:less PATH:/opt/homebrew/Cellar/php/8.2.8/bin:/opt/homebrew/opt/php@8.1/sbin:/opt/homebrew/opt/php@8.1/bin:/Users/milad.rahimi/bin:/Users/milad.rahimi/Scripts/ngrock:/Users/milad.rahimi/Scripts/google-cloud-sdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/milad.rahimi/Library/Application Support/JetBrains/Toolbox/scripts PWD:/Users/milad.rahimi/Projects/project SHELL:/bin/zsh SHLVL:1 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.8R8b4RkUYk/Listeners TERM:xterm-256color TERMINAL_EMULATOR:JetBrains-JediTerm TERM_SESSION_ID:06ef3b59-ab21-418c-9b87-0546ac49ff1b TMPDIR:/var/folders/b4/t_26_kdx7sq4x099mj49g4x00000gq/T/ USER:milad.rahimi XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/milad.rahimi/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.jetbrains.PhpStorm __CF_USER_TEXT_ENCODING:0x1F7:0x0:0x0]  subtask=-1 task=DevLoop
DEBU[0046] Found dependencies for dockerfile: [{composer.json /app true 4 4} {composer.lock /app true 4 4} {. /var/www/app true 79 79}]  subtask=-1 task=DevLoop
INFO[0046] files modified: [app/src/ControlApi/Handlers/MyHandler.php]  subtask=-1 task=DevLoop
DEBU[0046] Found dependencies for dockerfile: [{package-lock.json /base true 6 6} {package.json /base true 6 6} {. /base true 8 8}]  subtask=-1 task=DevLoop
DEBU[0046]  devloop: build false, sync true, deploy false  subtask=-1 task=DevLoop
Syncing 1 files for 12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:993c469437e918bb1946a2d38ea57f529f342a05d9c7f9665c956ef173881d8e
INFO[0046] Copying files:map[app/src/ControlApi/Handlers/MyHandler.php:[/var/www/app/src/ControlApi/Handlers/MyHandler.php]]to12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:993c469437e918bb1946a2d38ea57f529f342a05d9c7f9665c956ef173881d8e  subtask=-1 task=DevLoop
DEBU[0046] getting client config for kubeContext: `docker-desktop`  subtask=-1 task=DevLoop
DEBU[0046] Running command: [kubectl --context docker-desktop --namespace project exec project-backend-5978dbb769-zscnx --namespace project -c backend -i -- tar xmf - -C / --no-same-owner]  subtask=-1 task=DevLoop
DEBU[0046] Running command: [kubectl --context docker-desktop --namespace project exec project-sandbox-backend-78f8f9775c-zrmqh --namespace project -c backend -i -- tar xmf - -C / --no-same-owner]  subtask=-1 task=DevLoop
WARN[0046] Skipping deploy due to sync error:copying files: running [kubectl --context docker-desktop --namespace project exec project-sandbox-backend-78f8f9775c-zrmqh --namespace project -c backend -i -- tar xmf - -C / --no-same-owner]
 - stdout: ""
 - stderr: "tar: Removing leading `/' from member names\ntar: var/www/app/src/ControlApi/Handlers/MyHandler.php: Cannot open: File exists\ntar: Exiting with failure status due to previous errors\ncommand terminated with exit code 2\n"
 - cause: exit status 2: sync failed for artifact "12345.dkr.ecr.eu-central-1.amazonaws.com/project-dev/backend:993c469437e918bb1946a2d38ea57f529f342a05d9c7f9665c956ef173881d8e"  subtask=-1 task=DevLoop
Watching for changes...

As you can see, it updated project-backend-5978dbb769-zscnx container but failed to update project-sandbox-backend-78f8f9775c-zrmqh.